Jul 31 21:59:35.306874 ip-10-0-140-197 systemd[1]: Starting Kubernetes Kubelet... Jul 31 21:59:35.753765 ip-10-0-140-197 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 21:59:35.753765 ip-10-0-140-197 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 31 21:59:35.753765 ip-10-0-140-197 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 21:59:35.753765 ip-10-0-140-197 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 31 21:59:35.753765 ip-10-0-140-197 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 31 21:59:35.754931 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.754846 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 31 21:59:35.762396 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762382 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 21:59:35.762396 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762397 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762400 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762404 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762407 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762410 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762413 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762416 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762419 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762421 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762424 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762428 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762433 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762436 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762438 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762445 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762448 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762450 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762453 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762457 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 21:59:35.762463 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762461 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762465 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762468 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762470 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762473 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762476 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762479 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762482 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762484 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762487 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762489 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762492 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762494 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762497 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762499 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762502 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762504 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762507 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762509 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762512 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 21:59:35.762943 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762514 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762517 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762520 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762522 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762526 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762528 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762531 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762533 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762536 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762539 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762551 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762554 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762557 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762560 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762563 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762566 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762568 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762571 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762574 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762577 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 21:59:35.763427 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762579 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762582 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762584 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762587 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762589 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762592 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762604 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762607 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762610 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762612 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762615 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762618 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762620 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762623 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762626 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762628 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762631 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762635 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762638 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762640 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 21:59:35.764059 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762642 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762646 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762648 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762651 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762654 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762657 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762661 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762663 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762666 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762669 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762671 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762674 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762676 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762679 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762683 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762686 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762689 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762692 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.762695 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 21:59:35.764812 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763382 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763388 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763393 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763397 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763400 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763403 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763405 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763408 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763411 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763413 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763416 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763419 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763421 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763424 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763427 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763430 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763433 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763435 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763439 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763442 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 21:59:35.765364 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763445 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763448 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763451 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763453 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763456 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763458 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763461 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763464 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763466 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763469 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763471 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763474 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763476 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763479 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763482 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763485 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763487 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763489 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763492 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763494 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 21:59:35.765900 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763497 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763499 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763501 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763504 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763507 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763510 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763513 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763515 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763518 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763521 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763524 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763526 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763529 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763532 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763535 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763537 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763539 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763542 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763545 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763547 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 21:59:35.766395 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763549 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763552 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763554 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763557 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763559 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763562 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763564 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763567 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763569 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763571 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763574 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763576 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763579 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763581 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763584 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763587 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763590 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763594 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763612 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763615 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 21:59:35.766915 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763620 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763623 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763626 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763628 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763631 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763633 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763636 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763639 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763641 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763644 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763646 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763648 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763651 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763653 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763656 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763658 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763662 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763665 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.763667 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 21:59:35.767399 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765789 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765801 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765805 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765809 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765812 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765814 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765817 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765820 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765823 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765825 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765828 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765830 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765833 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765835 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765838 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765840 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765843 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765846 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765848 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765851 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 21:59:35.767883 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765854 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765860 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765862 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765865 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765868 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765870 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765874 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765878 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765881 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765884 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765887 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765889 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765892 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765896 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765899 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765902 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765905 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765909 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765912 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 21:59:35.768359 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765915 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765918 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765921 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765924 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765928 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765931 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765933 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765936 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765938 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765941 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765944 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765946 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765949 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765951 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765959 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765964 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765966 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765969 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765972 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765974 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 21:59:35.768827 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765977 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765979 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765982 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765985 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765987 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765990 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765994 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765997 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.765999 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766003 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766007 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766012 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766015 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766019 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766022 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766024 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766027 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766029 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766032 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 21:59:35.769319 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766034 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766037 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766039 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766042 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766045 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766048 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766050 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766053 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766055 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766058 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766060 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766063 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766065 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766068 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766070 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766073 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766075 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766078 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766081 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766084 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 21:59:35.769789 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766086 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766150 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766157 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766164 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766169 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766177 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766180 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766185 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766189 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766193 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766196 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766199 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766203 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766206 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766209 2554 flags.go:64] FLAG: --cgroup-root="" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766212 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766215 2554 flags.go:64] FLAG: --client-ca-file="" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766219 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766222 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766226 2554 flags.go:64] FLAG: --cluster-domain="" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766229 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766235 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766238 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766241 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 31 21:59:35.770287 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766245 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766248 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766251 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766255 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766258 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766261 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766264 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766267 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766270 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766273 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766277 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766280 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766283 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766286 2554 flags.go:64] FLAG: --enable-server="true" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766288 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766292 2554 flags.go:64] FLAG: --event-burst="100" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766296 2554 flags.go:64] FLAG: --event-qps="50" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766299 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766301 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766304 2554 flags.go:64] FLAG: --eviction-hard="" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766308 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766311 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766314 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766317 2554 flags.go:64] FLAG: --eviction-soft="" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766320 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 31 21:59:35.770908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766323 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766326 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766329 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766332 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766336 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766339 2554 flags.go:64] FLAG: --feature-gates="" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766342 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766345 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766348 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766351 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766355 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766358 2554 flags.go:64] FLAG: --help="false" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766361 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-140-197.ec2.internal" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766364 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766367 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766370 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766373 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766376 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766379 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766382 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766385 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766387 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766390 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766393 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 31 21:59:35.771533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766396 2554 flags.go:64] FLAG: --kube-reserved="" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766399 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766402 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766405 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766408 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766411 2554 flags.go:64] FLAG: --lock-file="" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766414 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766417 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766419 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766425 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766428 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766431 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766435 2554 flags.go:64] FLAG: --logging-format="text" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766437 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766441 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766443 2554 flags.go:64] FLAG: --manifest-url="" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766446 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766451 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766455 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766460 2554 flags.go:64] FLAG: --max-pods="110" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766463 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766466 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766468 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766471 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766474 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 31 21:59:35.772129 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766477 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766480 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766487 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766490 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766493 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766496 2554 flags.go:64] FLAG: --pod-cidr="" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766499 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766504 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766507 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766510 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766513 2554 flags.go:64] FLAG: --port="10250" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766516 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766519 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0edc964308d40ea86" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766522 2554 flags.go:64] FLAG: --qos-reserved="" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766525 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766528 2554 flags.go:64] FLAG: --register-node="true" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766531 2554 flags.go:64] FLAG: --register-with-taints="" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766534 2554 flags.go:64] FLAG: --registry-burst="10" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766537 2554 flags.go:64] FLAG: --registry-qps="5" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766540 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766543 2554 flags.go:64] FLAG: --reserved-memory="" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766547 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766550 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766552 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766555 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 31 21:59:35.772743 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766558 2554 flags.go:64] FLAG: --runonce="false" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766562 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766565 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766568 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766571 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766574 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766577 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766580 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766583 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766585 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766588 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766591 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766594 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766614 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766619 2554 flags.go:64] FLAG: --system-cgroups="" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766623 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766631 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766634 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766637 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766641 2554 flags.go:64] FLAG: --tls-min-version="" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766644 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766647 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766650 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766653 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766655 2554 flags.go:64] FLAG: --v="2" Jul 31 21:59:35.773376 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766660 2554 flags.go:64] FLAG: --version="false" Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766666 2554 flags.go:64] FLAG: --vmodule="" Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766669 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.766673 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766788 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766791 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766795 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766799 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766804 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766807 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766809 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766812 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766815 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766817 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766820 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766822 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766825 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766828 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766830 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766833 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 21:59:35.774108 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766835 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766838 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766842 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766845 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766848 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766851 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766855 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766858 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766860 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766863 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766865 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766868 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766871 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766873 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766876 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766879 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766881 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766884 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766886 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766889 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 21:59:35.774641 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766893 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766895 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766898 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766901 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766903 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766907 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766910 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766914 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766917 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766919 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766922 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766925 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766928 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766931 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766934 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766936 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766939 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766943 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766946 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 21:59:35.775127 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766948 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766951 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766953 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766956 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766959 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766962 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766964 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766966 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766969 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766972 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766974 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766977 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766979 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766983 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766985 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766988 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766991 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766993 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766995 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.766998 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 21:59:35.775590 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767001 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767003 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767006 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767009 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767011 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767014 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767017 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767020 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767022 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767025 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767029 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767032 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767034 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767037 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767039 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767042 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767045 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767047 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767049 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767052 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 21:59:35.776105 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767054 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767057 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767060 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.767063 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.767780 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.774613 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.774628 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.774645 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.774651 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774719 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774724 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774727 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774730 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774733 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 21:59:35.776593 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774736 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774739 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774742 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774745 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774748 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774751 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774754 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774756 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774759 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774761 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774764 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774767 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774769 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774772 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774775 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774777 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774780 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774782 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774785 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774794 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 21:59:35.776957 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774797 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774800 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774802 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774804 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774808 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774811 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774814 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774816 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774819 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774822 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774824 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774827 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774829 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774832 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774835 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774837 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774839 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774842 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774844 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774847 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 21:59:35.777431 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774850 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774852 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774855 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774858 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774860 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774863 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774865 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774868 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774870 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774873 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774875 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774879 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774883 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774886 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774889 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774891 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774895 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774897 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774900 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774903 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 21:59:35.777977 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774905 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774908 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774910 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774913 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774915 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774918 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774920 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774923 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774925 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774929 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774933 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774937 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774940 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774943 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774945 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774948 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774950 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774953 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774956 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 21:59:35.778467 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774958 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774961 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774964 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774967 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774970 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774972 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774977 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774980 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774983 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774987 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774989 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774992 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774994 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774997 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.774999 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 21:59:35.778947 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.775004 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775116 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775121 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775125 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775128 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775131 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775133 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775136 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775139 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775142 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775144 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775147 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775149 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775151 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775154 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775157 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775159 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775162 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775164 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775167 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 31 21:59:35.779310 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775170 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775172 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775174 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775178 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775182 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775185 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775188 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775191 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775194 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775196 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775199 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775202 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775204 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775206 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775209 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775211 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775214 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775216 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775218 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 31 21:59:35.779855 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775221 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775223 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775226 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775229 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775231 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775233 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775236 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775238 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775241 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775243 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775246 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775248 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775251 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775253 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775256 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775258 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775261 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775263 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775267 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775269 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 31 21:59:35.780313 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775272 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775274 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775277 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775280 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775282 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775285 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775287 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775290 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775292 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775295 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775297 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775300 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775302 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775305 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775307 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775309 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775312 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775314 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775317 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775319 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 31 21:59:35.780802 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775322 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775324 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775326 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775329 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775331 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775334 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775336 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775338 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775341 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775343 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775346 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775350 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775352 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775355 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775358 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775362 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775364 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775367 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775369 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 31 21:59:35.781278 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775373 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:35.775376 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.775381 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.775475 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.777537 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.779291 2554 server.go:1019] "Starting client certificate rotation" Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.779401 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 21:59:35.781747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.779434 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 31 21:59:35.782154 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.782140 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 21:59:35.787070 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.787049 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 31 21:59:35.799397 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.799381 2554 log.go:25] "Validated CRI v1 runtime API" Jul 31 21:59:35.801796 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.801781 2554 log.go:25] "Validated CRI v1 image API" Jul 31 21:59:35.803378 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.803362 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 31 21:59:35.808214 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.808195 2554 fs.go:135] Filesystem UUIDs: map[1711d00b-6a80-43fc-8532-4a3657372be8:/dev/nvme0n1p4 3531ecdf-3ffb-4854-a2ba-215cafc94b37:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 31 21:59:35.808293 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.808212 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 31 21:59:35.812332 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.812315 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 21:59:35.815310 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.815200 2554 manager.go:216] Machine: {Timestamp:2026-07-31 21:59:35.813660723 +0000 UTC m=+0.378509296 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099645 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2950a15a4d0d0a1e59a1345edcbace SystemUUID:ec2950a1-5a4d-0d0a-1e59-a1345edcbace BootID:4ebd545e-acc3-48a5-9ebb-e741e4024eea Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:a4:64:f1:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:a4:64:f1:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:6e:56:40:c4:7a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 31 21:59:35.815310 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.815308 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 31 21:59:35.815437 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.815385 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 31 21:59:35.819048 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.819023 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 31 21:59:35.819172 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.819050 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-197.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 31 21:59:35.819877 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.819868 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 31 21:59:35.819908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.819880 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 31 21:59:35.819908 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.819888 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 21:59:35.819967 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.819920 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 31 21:59:35.821628 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.821618 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 31 21:59:35.826457 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826446 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 31 21:59:35.826568 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826558 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 31 21:59:35.826620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826589 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 31 21:59:35.826620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826613 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 31 21:59:35.826694 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826627 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 31 21:59:35.826694 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826636 2554 kubelet.go:398] "Adding apiserver pod source" Jul 31 21:59:35.826694 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826644 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 31 21:59:35.826979 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826967 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 21:59:35.827026 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.826987 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 31 21:59:35.830208 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.830188 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 31 21:59:35.832068 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832053 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 31 21:59:35.832135 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832083 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 31 21:59:35.832918 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832907 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832923 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832930 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832935 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832943 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832949 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832957 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832962 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832968 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 31 21:59:35.832972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832974 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 31 21:59:35.833211 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832982 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 31 21:59:35.833211 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.832990 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 31 21:59:35.834752 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.834737 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 31 21:59:35.834800 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.834755 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 31 21:59:35.837473 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.837451 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-197.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 31 21:59:35.837473 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.837460 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 31 21:59:35.837582 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.837474 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-197.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 31 21:59:35.837795 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.837783 2554 server.go:1268] "Started kubelet" Jul 31 21:59:35.837875 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.837842 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 31 21:59:35.837928 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.837866 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 31 21:59:35.837988 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.837934 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 31 21:59:35.838683 ip-10-0-140-197 systemd[1]: Started Kubernetes Kubelet. Jul 31 21:59:35.839329 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.839312 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 31 21:59:35.839912 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.839901 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 31 21:59:35.843968 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.843940 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 31 21:59:35.847665 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.847645 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 31 21:59:35.847778 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.847647 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 31 21:59:35.847778 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.847727 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:35.848510 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848493 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 31 21:59:35.848510 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848500 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 31 21:59:35.848679 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848522 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 31 21:59:35.848679 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848654 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 31 21:59:35.848679 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848666 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 31 21:59:35.848909 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848892 2554 factory.go:153] Registering CRI-O factory Jul 31 21:59:35.848909 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848910 2554 factory.go:223] Registration of the crio container factory successfully Jul 31 21:59:35.849054 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848961 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 31 21:59:35.849054 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848970 2554 factory.go:55] Registering systemd factory Jul 31 21:59:35.849054 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848977 2554 factory.go:223] Registration of the systemd container factory successfully Jul 31 21:59:35.849054 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.848996 2554 factory.go:103] Registering Raw factory Jul 31 21:59:35.849054 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.849010 2554 manager.go:1196] Started watching for new ooms in manager Jul 31 21:59:35.849477 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.849462 2554 manager.go:349] Starting recovery of all containers Jul 31 21:59:35.852114 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.852084 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 31 21:59:35.852219 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.852192 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-197.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 31 21:59:35.855802 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.852102 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-197.ec2.internal.18c77ec841c623a0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-197.ec2.internal,UID:ip-10-0-140-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-197.ec2.internal,},FirstTimestamp:2026-07-31 21:59:35.837754272 +0000 UTC m=+0.402602845,LastTimestamp:2026-07-31 21:59:35.837754272 +0000 UTC m=+0.402602845,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-197.ec2.internal,}" Jul 31 21:59:35.858001 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.857978 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6r85n" Jul 31 21:59:35.862952 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.862937 2554 manager.go:354] Recovery completed Jul 31 21:59:35.864179 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.864114 2554 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 31 21:59:35.865651 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.865631 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6r85n" Jul 31 21:59:35.867037 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.867025 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:35.868792 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.868774 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:35.868876 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.868801 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:35.868876 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.868811 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:35.869325 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.869297 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 31 21:59:35.869325 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.869313 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 31 21:59:35.869440 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.869332 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 31 21:59:35.869440 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.869314 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:35.870206 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.870181 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:35.870284 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.870212 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:35.870284 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.870227 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:35.870942 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.870881 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-197.ec2.internal.18c77ec8439faeed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-197.ec2.internal,UID:ip-10-0-140-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-197.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-197.ec2.internal,},FirstTimestamp:2026-07-31 21:59:35.868788461 +0000 UTC m=+0.433637033,LastTimestamp:2026-07-31 21:59:35.868788461 +0000 UTC m=+0.433637033,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-197.ec2.internal,}" Jul 31 21:59:35.871765 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.871753 2554 policy_none.go:49] "None policy: Start" Jul 31 21:59:35.871803 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.871769 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 31 21:59:35.871803 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.871779 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 31 21:59:35.873716 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.873705 2554 policy_none.go:47] "Start" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914164 2554 manager.go:337] "Starting Device Plugin manager" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.914194 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914203 2554 server.go:87] "Starting device plugin registration server" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914460 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914484 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914584 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914685 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.914694 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.915246 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 31 21:59:35.925320 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.915285 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:35.949093 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.949065 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 31 21:59:35.950266 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.950246 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 31 21:59:35.950342 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.950284 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 31 21:59:35.950342 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.950312 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 31 21:59:35.950442 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:35.950348 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 31 21:59:35.953151 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:35.953134 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 21:59:36.015634 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.015561 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:36.016606 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.016580 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:36.016668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.016626 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:36.016668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.016638 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:36.016668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.016659 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.028880 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.028860 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.028928 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.028884 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-197.ec2.internal\": node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.050751 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.050580 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal"] Jul 31 21:59:36.050833 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.050797 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:36.052127 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.052107 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:36.052200 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.052136 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:36.052200 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.052146 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:36.054533 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.054521 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:36.054679 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.054653 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.054717 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.054694 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:36.055266 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.055250 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:36.055266 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.055261 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:36.055359 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.055279 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:36.055359 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.055294 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:36.055359 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.055280 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:36.055482 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.055385 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:36.055894 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.055868 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.057430 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.057414 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.057509 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.057439 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 31 21:59:36.058462 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.058449 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientMemory" Jul 31 21:59:36.058523 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.058474 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasNoDiskPressure" Jul 31 21:59:36.058523 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.058483 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeHasSufficientPID" Jul 31 21:59:36.079169 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.079149 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-197.ec2.internal\" not found" node="ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.083626 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.083611 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-197.ec2.internal\" not found" node="ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.150298 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.150271 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a563827a727e1fa7ade2716ea0dbf4d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal\" (UID: \"a563827a727e1fa7ade2716ea0dbf4d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.150298 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.150298 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a563827a727e1fa7ade2716ea0dbf4d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal\" (UID: \"a563827a727e1fa7ade2716ea0dbf4d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.150499 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.150318 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/01e690b2a67d9f68121efde86b8c3543-config\") pod \"kube-apiserver-proxy-ip-10-0-140-197.ec2.internal\" (UID: \"01e690b2a67d9f68121efde86b8c3543\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.156374 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.156357 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.250785 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.250754 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a563827a727e1fa7ade2716ea0dbf4d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal\" (UID: \"a563827a727e1fa7ade2716ea0dbf4d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.250911 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.250801 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/01e690b2a67d9f68121efde86b8c3543-config\") pod \"kube-apiserver-proxy-ip-10-0-140-197.ec2.internal\" (UID: \"01e690b2a67d9f68121efde86b8c3543\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.250911 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.250826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a563827a727e1fa7ade2716ea0dbf4d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal\" (UID: \"a563827a727e1fa7ade2716ea0dbf4d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.250911 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.250853 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a563827a727e1fa7ade2716ea0dbf4d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal\" (UID: \"a563827a727e1fa7ade2716ea0dbf4d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.251010 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.250855 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a563827a727e1fa7ade2716ea0dbf4d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal\" (UID: \"a563827a727e1fa7ade2716ea0dbf4d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.251010 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.250859 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/01e690b2a67d9f68121efde86b8c3543-config\") pod \"kube-apiserver-proxy-ip-10-0-140-197.ec2.internal\" (UID: \"01e690b2a67d9f68121efde86b8c3543\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.256861 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.256846 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.357698 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.357627 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.380760 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.380740 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.386871 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.386853 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" Jul 31 21:59:36.458847 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.458813 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.559102 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.559072 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.660200 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.660129 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.760533 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.760486 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.779021 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.778992 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 31 21:59:36.779210 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.779190 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 31 21:59:36.854010 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.853981 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 21:59:36.861110 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.861087 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:36.871294 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.871264 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 21:54:36 +0000 UTC" deadline="2028-02-11 14:28:23.021059975 +0000 UTC" Jul 31 21:59:36.871294 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:36.871291 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13432h28m46.149771695s" Jul 31 21:59:36.961657 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:36.961575 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:37.062725 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.062691 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:37.163760 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.163730 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-197.ec2.internal\" not found" Jul 31 21:59:37.172918 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:37.172884 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e690b2a67d9f68121efde86b8c3543.slice/crio-1760d100a4bc014f774ba5b2a535a9c1858b2ddce05f3d53240e7f61c2d9f436 WatchSource:0}: Error finding container 1760d100a4bc014f774ba5b2a535a9c1858b2ddce05f3d53240e7f61c2d9f436: Status 404 returned error can't find the container with id 1760d100a4bc014f774ba5b2a535a9c1858b2ddce05f3d53240e7f61c2d9f436 Jul 31 21:59:37.173508 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:37.173491 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda563827a727e1fa7ade2716ea0dbf4d5.slice/crio-8b20ec609b43cd4eac63e5882995fbc516c8b928ae3a37077f694436d8cdf081 WatchSource:0}: Error finding container 8b20ec609b43cd4eac63e5882995fbc516c8b928ae3a37077f694436d8cdf081: Status 404 returned error can't find the container with id 8b20ec609b43cd4eac63e5882995fbc516c8b928ae3a37077f694436d8cdf081 Jul 31 21:59:37.180674 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.180653 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 21:59:37.207942 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.207919 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 21:59:37.251305 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.251232 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 21:59:37.348846 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.348807 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" Jul 31 21:59:37.367945 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.367927 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 21:59:37.369706 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.369692 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" Jul 31 21:59:37.388212 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.388188 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 31 21:59:37.683823 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.683741 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 31 21:59:37.828158 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.828120 2554 apiserver.go:52] "Watching apiserver" Jul 31 21:59:37.835313 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.835284 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 31 21:59:37.835824 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.835797 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf","openshift-dns/node-resolver-xhptq","openshift-multus/multus-additional-cni-plugins-zwxm5","openshift-network-operator/iptables-alerter-cr798","openshift-ovn-kubernetes/ovnkube-node-fmtfd","kube-system/global-pull-secret-syncer-hkdcf","kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal","openshift-cluster-node-tuning-operator/tuned-d7vlw","openshift-image-registry/node-ca-vt8r7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal","openshift-multus/multus-b8nxl","openshift-multus/network-metrics-daemon-qqgnd","openshift-network-diagnostics/network-check-target-8nw5p","kube-system/konnectivity-agent-5rxkg"] Jul 31 21:59:37.838615 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.838588 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:37.838739 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.838696 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:37.840995 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.840975 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.843401 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.843382 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.845128 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.845106 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.845231 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.845113 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.845315 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.845294 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c7g8f\"" Jul 31 21:59:37.845703 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.845683 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.848033 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848011 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.848407 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848390 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848563 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848718 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848748 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848843 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848842 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848886 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848857 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848939 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q9wg2\"" Jul 31 21:59:37.849020 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.848982 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5f278\"" Jul 31 21:59:37.850529 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.850508 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.853658 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.852537 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.853658 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.852637 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:37.853658 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.853243 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xjfr2\"" Jul 31 21:59:37.853658 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.853614 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.855764 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.855739 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.855941 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.855918 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 31 21:59:37.856060 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.855964 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 31 21:59:37.856060 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.855923 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.856444 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.856422 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 31 21:59:37.856444 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.856437 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 31 21:59:37.858321 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.858301 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.859476 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.859457 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.859654 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.859615 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.859747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.859734 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5mm64\"" Jul 31 21:59:37.859845 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.859828 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 31 21:59:37.861097 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861078 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861430 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-node-log\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861468 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2ed8dd22-e69c-44e5-8184-1694658e7c85-hosts-file\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861496 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gswzc\" (UniqueName: \"kubernetes.io/projected/2ed8dd22-e69c-44e5-8184-1694658e7c85-kube-api-access-gswzc\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861521 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-system-cni-dir\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861546 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861571 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861601 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861626 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovnkube-script-lib\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861681 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed8dd22-e69c-44e5-8184-1694658e7c85-tmp-dir\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861731 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-os-release\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861755 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861780 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-log-socket\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861806 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861835 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:37.861953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861868 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-host-slash\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861891 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-kubelet\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861914 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-etc-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.861977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovnkube-config\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862032 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4jgr\" (UniqueName: \"kubernetes.io/projected/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-kube-api-access-d4jgr\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862071 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-cni-bin\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862114 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cnibin\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862143 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862169 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-iptables-alerter-script\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-slash\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-run-netns\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862257 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-var-lib-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862291 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-env-overrides\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862327 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovn-node-metrics-cert\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862377 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-systemd-units\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862412 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-systemd\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.862858 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862460 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-579cj\" (UniqueName: \"kubernetes.io/projected/5166b1fa-37db-4e80-8424-5c0f4ece91a3-kube-api-access-579cj\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.863620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862497 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.863620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862538 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-ovn\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.863620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862565 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94br2\" (UniqueName: \"kubernetes.io/projected/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-kube-api-access-94br2\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.863620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862591 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-cni-netd\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.863620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862632 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a473fec7-53b1-4440-b7cf-5cdaf810c897-kubelet-config\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.863620 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.862654 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a473fec7-53b1-4440-b7cf-5cdaf810c897-dbus\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.863970 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.863645 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.863970 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.863945 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 31 21:59:37.864076 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.863989 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 31 21:59:37.864076 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.864068 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.864180 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.864156 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-csph2\"" Jul 31 21:59:37.864377 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.864356 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 31 21:59:37.864697 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.864629 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mcjc6\"" Jul 31 21:59:37.864854 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.864655 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:37.868222 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.868110 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.869406 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.869385 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7wdq9\"" Jul 31 21:59:37.869729 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.869709 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 31 21:59:37.869872 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.869715 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 31 21:59:37.871077 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.871055 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:37.871171 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.871147 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:37.873883 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.873855 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 31 21:59:37.874512 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.874454 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pdjhc\"" Jul 31 21:59:37.875968 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.875946 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 31 21:59:37.950125 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.950044 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 31 21:59:37.962877 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.962846 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-ovn\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.963117 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963099 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-etc-selinux\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.963223 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963209 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-sys\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.963324 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963311 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d67851f3-1f5a-44f0-838e-0279d191776c-agent-certs\") pod \"konnectivity-agent-5rxkg\" (UID: \"d67851f3-1f5a-44f0-838e-0279d191776c\") " pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:37.963452 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963435 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-conf-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.963565 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963551 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94br2\" (UniqueName: \"kubernetes.io/projected/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-kube-api-access-94br2\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.963662 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963649 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-cni-netd\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.963862 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963843 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a473fec7-53b1-4440-b7cf-5cdaf810c897-kubelet-config\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.963972 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.963958 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-socket-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.964073 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p99q9\" (UniqueName: \"kubernetes.io/projected/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-kube-api-access-p99q9\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.964192 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964178 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-system-cni-dir\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.964302 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964288 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovnkube-script-lib\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.964413 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964392 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysctl-d\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.964511 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964498 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysctl-conf\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.964605 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964592 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-systemd\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.964725 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964709 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.964839 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964813 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.964998 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964971 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-serviceca\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:37.965112 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965096 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptt5l\" (UniqueName: \"kubernetes.io/projected/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-kube-api-access-ptt5l\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:37.965216 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965200 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-socket-dir-parent\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.965337 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965310 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-system-cni-dir\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.965469 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965451 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-daemon-config\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.965594 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965569 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-host-slash\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.965594 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965605 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-kubelet\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.965793 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965633 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovnkube-config\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.965793 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965660 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-device-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.965793 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965734 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqgdk\" (UniqueName: \"kubernetes.io/projected/0789200e-8da2-45b2-be9d-1939083a0505-kube-api-access-qqgdk\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.965793 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965765 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-cni-bin\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966000 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965795 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-var-lib-kubelet\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.966000 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965822 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-netns\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.966000 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965853 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.966000 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965904 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-iptables-alerter-script\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.966000 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965924 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966292 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966112 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.966292 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964821 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a473fec7-53b1-4440-b7cf-5cdaf810c897-kubelet-config\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.966292 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.964873 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-ovn\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966292 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.965232 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-cni-netd\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966292 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966243 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-host-slash\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.966292 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966287 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-kubelet\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966563 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966493 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.966829 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966799 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovnkube-config\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966897 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966880 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-cni-bin\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966946 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-slash\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.966946 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966934 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovn-node-metrics-cert\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.967047 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966951 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-kubernetes\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.967047 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966966 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0789200e-8da2-45b2-be9d-1939083a0505-etc-tuned\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.967047 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.966980 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-cni-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.967047 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967004 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-systemd-units\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.967047 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967029 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-579cj\" (UniqueName: \"kubernetes.io/projected/5166b1fa-37db-4e80-8424-5c0f4ece91a3-kube-api-access-579cj\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967046 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-iptables-alerter-script\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967052 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-registration-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967083 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-slash\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967105 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-cnibin\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967133 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnsdk\" (UniqueName: \"kubernetes.io/projected/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-kube-api-access-cnsdk\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a473fec7-53b1-4440-b7cf-5cdaf810c897-dbus\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967192 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6vv\" (UniqueName: \"kubernetes.io/projected/17e24bb8-7793-491d-b095-af57702b62af-kube-api-access-9m6vv\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967229 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-lib-modules\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967253 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-host\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.967308 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967278 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-host\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:37.967815 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967318 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d67851f3-1f5a-44f0-838e-0279d191776c-konnectivity-ca\") pod \"konnectivity-agent-5rxkg\" (UID: \"d67851f3-1f5a-44f0-838e-0279d191776c\") " pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:37.967815 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967346 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-cni-bin\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.967815 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967372 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-node-log\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.967815 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967399 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2ed8dd22-e69c-44e5-8184-1694658e7c85-hosts-file\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.967815 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967407 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 31 21:59:37.968209 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.968187 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovnkube-script-lib\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.968393 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.968347 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" event={"ID":"01e690b2a67d9f68121efde86b8c3543","Type":"ContainerStarted","Data":"1760d100a4bc014f774ba5b2a535a9c1858b2ddce05f3d53240e7f61c2d9f436"} Jul 31 21:59:37.968453 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.967429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gswzc\" (UniqueName: \"kubernetes.io/projected/2ed8dd22-e69c-44e5-8184-1694658e7c85-kube-api-access-gswzc\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.968670 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a473fec7-53b1-4440-b7cf-5cdaf810c897-dbus\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.968785 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-systemd-units\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.968825 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-node-log\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969069 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2ed8dd22-e69c-44e5-8184-1694658e7c85-hosts-file\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969108 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969138 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-modprobe-d\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969228 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-kubelet\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed8dd22-e69c-44e5-8184-1694658e7c85-tmp-dir\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969283 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-os-release\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969307 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-log-socket\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969332 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-sys-fs\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969381 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0789200e-8da2-45b2-be9d-1939083a0505-tmp\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969404 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-cni-multus\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.972241 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969427 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-hostroot\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969456 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969483 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-etc-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969511 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969558 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysconfig\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969582 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-run\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969608 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-cni-binary-copy\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969634 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-multus-certs\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.969667 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4jgr\" (UniqueName: \"kubernetes.io/projected/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-kube-api-access-d4jgr\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971704 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-k8s-cni-cncf-io\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971734 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-etc-kubernetes\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971764 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cnibin\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971790 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-run-netns\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971827 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-var-lib-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971855 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-env-overrides\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971883 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-system-cni-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971905 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5166b1fa-37db-4e80-8424-5c0f4ece91a3-ovn-node-metrics-cert\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.973218 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.970170 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.970587 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971909 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-systemd\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.970632 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971988 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972016 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-cnibin\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972020 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-os-release\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.970920 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed8dd22-e69c-44e5-8184-1694658e7c85-tmp-dir\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972047 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.970979 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-os-release\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971013 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-log-socket\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971045 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.971077 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-etc-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972149 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-run-systemd\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972164 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-host-run-netns\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.972268 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.972340 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:38.472314607 +0000 UTC m=+3.037163173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:37.974042 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972542 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5166b1fa-37db-4e80-8424-5c0f4ece91a3-var-lib-openvswitch\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974982 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.972585 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5166b1fa-37db-4e80-8424-5c0f4ece91a3-env-overrides\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.974982 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.973734 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" event={"ID":"a563827a727e1fa7ade2716ea0dbf4d5","Type":"ContainerStarted","Data":"8b20ec609b43cd4eac63e5882995fbc516c8b928ae3a37077f694436d8cdf081"} Jul 31 21:59:37.979897 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.979533 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 21:59:37.979897 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.979574 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 21:59:37.979897 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.979588 2554 projected.go:196] Error preparing data for projected volume kube-api-access-p9lvd for pod openshift-network-diagnostics/network-check-target-8nw5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:37.979897 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:37.979693 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd podName:efe077f6-5e11-4e9a-bb32-b6f200b4fd76 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:38.479658773 +0000 UTC m=+3.044507337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p9lvd" (UniqueName: "kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd") pod "network-check-target-8nw5p" (UID: "efe077f6-5e11-4e9a-bb32-b6f200b4fd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:37.981166 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.981139 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-579cj\" (UniqueName: \"kubernetes.io/projected/5166b1fa-37db-4e80-8424-5c0f4ece91a3-kube-api-access-579cj\") pod \"ovnkube-node-fmtfd\" (UID: \"5166b1fa-37db-4e80-8424-5c0f4ece91a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:37.981497 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.981477 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gswzc\" (UniqueName: \"kubernetes.io/projected/2ed8dd22-e69c-44e5-8184-1694658e7c85-kube-api-access-gswzc\") pod \"node-resolver-xhptq\" (UID: \"2ed8dd22-e69c-44e5-8184-1694658e7c85\") " pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:37.984135 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.984110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4jgr\" (UniqueName: \"kubernetes.io/projected/7ee82f9d-1e38-4eb9-9997-53f53c456c1d-kube-api-access-d4jgr\") pod \"iptables-alerter-cr798\" (UID: \"7ee82f9d-1e38-4eb9-9997-53f53c456c1d\") " pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:37.984659 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:37.984611 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94br2\" (UniqueName: \"kubernetes.io/projected/790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0-kube-api-access-94br2\") pod \"multus-additional-cni-plugins-zwxm5\" (UID: \"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0\") " pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:38.020381 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.020345 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wj2gc" Jul 31 21:59:38.030897 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.030860 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wj2gc" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073253 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-registration-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073346 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-registration-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073401 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-cnibin\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073435 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnsdk\" (UniqueName: \"kubernetes.io/projected/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-kube-api-access-cnsdk\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073467 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6vv\" (UniqueName: \"kubernetes.io/projected/17e24bb8-7793-491d-b095-af57702b62af-kube-api-access-9m6vv\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073496 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-lib-modules\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073521 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-host\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073544 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-host\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073571 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d67851f3-1f5a-44f0-838e-0279d191776c-konnectivity-ca\") pod \"konnectivity-agent-5rxkg\" (UID: \"d67851f3-1f5a-44f0-838e-0279d191776c\") " pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073597 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-cni-bin\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073648 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-modprobe-d\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073675 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-kubelet\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073705 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-sys-fs\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073755 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0789200e-8da2-45b2-be9d-1939083a0505-tmp\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073763 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-cni-bin\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073793 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-cni-multus\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.074370 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073848 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-hostroot\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073868 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-cni-multus\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073924 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-cnibin\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073926 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysconfig\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-run\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073976 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-modprobe-d\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073991 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-cni-binary-copy\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074005 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-sys-fs\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074020 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-multus-certs\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074044 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-host\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074060 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-multus-certs\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074061 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-k8s-cni-cncf-io\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074101 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-k8s-cni-cncf-io\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074104 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-etc-kubernetes\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074142 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-system-cni-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074141 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-etc-kubernetes\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074191 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-os-release\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.075668 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074198 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-hostroot\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074224 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074255 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-etc-selinux\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074264 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-os-release\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074279 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-sys\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074307 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d67851f3-1f5a-44f0-838e-0279d191776c-agent-certs\") pod \"konnectivity-agent-5rxkg\" (UID: \"d67851f3-1f5a-44f0-838e-0279d191776c\") " pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074314 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-system-cni-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073759 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-var-lib-kubelet\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074339 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-conf-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074405 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-socket-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.074424 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074432 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p99q9\" (UniqueName: \"kubernetes.io/projected/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-kube-api-access-p99q9\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074457 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysctl-d\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.074487 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 21:59:38.574468018 +0000 UTC m=+3.139316593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074508 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysctl-conf\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-systemd\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074566 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-serviceca\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.076648 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074585 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysctl-d\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074595 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptt5l\" (UniqueName: \"kubernetes.io/projected/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-kube-api-access-ptt5l\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074624 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-socket-dir-parent\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074632 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d67851f3-1f5a-44f0-838e-0279d191776c-konnectivity-ca\") pod \"konnectivity-agent-5rxkg\" (UID: \"d67851f3-1f5a-44f0-838e-0279d191776c\") " pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074649 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-daemon-config\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074698 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-device-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074716 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-host\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074721 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-socket-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074723 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqgdk\" (UniqueName: \"kubernetes.io/projected/0789200e-8da2-45b2-be9d-1939083a0505-kube-api-access-qqgdk\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074856 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-cni-binary-copy\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.073899 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-lib-modules\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-var-lib-kubelet\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.074960 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-etc-selinux\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075034 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-device-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075046 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysconfig\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075092 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-sysctl-conf\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-socket-dir-parent\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075142 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-netns\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.077570 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-conf-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075184 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-run\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075224 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-sys\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075199 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-var-lib-kubelet\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075273 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-systemd\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075275 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17e24bb8-7793-491d-b095-af57702b62af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075317 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-kubernetes\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075332 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-host-run-netns\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075449 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0789200e-8da2-45b2-be9d-1939083a0505-etc-tuned\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-cni-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075534 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-daemon-config\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075636 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-multus-cni-dir\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075653 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0789200e-8da2-45b2-be9d-1939083a0505-etc-kubernetes\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.075809 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-serviceca\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.078513 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.078439 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0789200e-8da2-45b2-be9d-1939083a0505-etc-tuned\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.079498 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.079445 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0789200e-8da2-45b2-be9d-1939083a0505-tmp\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.080732 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.080699 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d67851f3-1f5a-44f0-838e-0279d191776c-agent-certs\") pod \"konnectivity-agent-5rxkg\" (UID: \"d67851f3-1f5a-44f0-838e-0279d191776c\") " pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:38.089906 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.089851 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqgdk\" (UniqueName: \"kubernetes.io/projected/0789200e-8da2-45b2-be9d-1939083a0505-kube-api-access-qqgdk\") pod \"tuned-d7vlw\" (UID: \"0789200e-8da2-45b2-be9d-1939083a0505\") " pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.090267 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.090237 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6vv\" (UniqueName: \"kubernetes.io/projected/17e24bb8-7793-491d-b095-af57702b62af-kube-api-access-9m6vv\") pod \"aws-ebs-csi-driver-node-42cpf\" (UID: \"17e24bb8-7793-491d-b095-af57702b62af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.091521 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.091495 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnsdk\" (UniqueName: \"kubernetes.io/projected/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-kube-api-access-cnsdk\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:38.092103 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.092054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p99q9\" (UniqueName: \"kubernetes.io/projected/6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418-kube-api-access-p99q9\") pod \"multus-b8nxl\" (UID: \"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418\") " pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.094306 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.094255 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptt5l\" (UniqueName: \"kubernetes.io/projected/b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7-kube-api-access-ptt5l\") pod \"node-ca-vt8r7\" (UID: \"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7\") " pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.153831 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.153794 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xhptq" Jul 31 21:59:38.162052 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.162025 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" Jul 31 21:59:38.172805 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:38.172765 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed8dd22_e69c_44e5_8184_1694658e7c85.slice/crio-6c6a1778d9cd55383ef9f11dee366160921573b1bf1e8502f729380c24495a2e WatchSource:0}: Error finding container 6c6a1778d9cd55383ef9f11dee366160921573b1bf1e8502f729380c24495a2e: Status 404 returned error can't find the container with id 6c6a1778d9cd55383ef9f11dee366160921573b1bf1e8502f729380c24495a2e Jul 31 21:59:38.172927 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.172887 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cr798" Jul 31 21:59:38.181861 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.181551 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 21:59:38.184984 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:38.184950 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee82f9d_1e38_4eb9_9997_53f53c456c1d.slice/crio-c59be5574e94ae00b4076349a73b6de18ba20106064593fa2cb2e9864d3912a3 WatchSource:0}: Error finding container c59be5574e94ae00b4076349a73b6de18ba20106064593fa2cb2e9864d3912a3: Status 404 returned error can't find the container with id c59be5574e94ae00b4076349a73b6de18ba20106064593fa2cb2e9864d3912a3 Jul 31 21:59:38.190719 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.190694 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" Jul 31 21:59:38.193736 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:38.193708 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5166b1fa_37db_4e80_8424_5c0f4ece91a3.slice/crio-42f132fc708ccee09417590f734181eb39fe79246e02261db11683b23aab3a15 WatchSource:0}: Error finding container 42f132fc708ccee09417590f734181eb39fe79246e02261db11683b23aab3a15: Status 404 returned error can't find the container with id 42f132fc708ccee09417590f734181eb39fe79246e02261db11683b23aab3a15 Jul 31 21:59:38.198730 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.198707 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" Jul 31 21:59:38.208602 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.208576 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vt8r7" Jul 31 21:59:38.214290 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:38.214261 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0789200e_8da2_45b2_be9d_1939083a0505.slice/crio-67122ff478ea7b25ea8ddce9d429f8198f0c02d002473e3cc469e76d8e58ed18 WatchSource:0}: Error finding container 67122ff478ea7b25ea8ddce9d429f8198f0c02d002473e3cc469e76d8e58ed18: Status 404 returned error can't find the container with id 67122ff478ea7b25ea8ddce9d429f8198f0c02d002473e3cc469e76d8e58ed18 Jul 31 21:59:38.217062 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.216622 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:38.225004 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.224836 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b8nxl" Jul 31 21:59:38.235579 ip-10-0-140-197 kubenswrapper[2554]: W0731 21:59:38.235553 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d90bcf9_d9dd_4c3f_a5b8_a5e0de58c418.slice/crio-e7e1ccadb6991db9fd226dd49c4157a59adbe4753da33973ade17adb905f035e WatchSource:0}: Error finding container e7e1ccadb6991db9fd226dd49c4157a59adbe4753da33973ade17adb905f035e: Status 404 returned error can't find the container with id e7e1ccadb6991db9fd226dd49c4157a59adbe4753da33973ade17adb905f035e Jul 31 21:59:38.479257 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.479157 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:38.479434 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.479311 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:38.479434 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.479383 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:39.479367045 +0000 UTC m=+4.044215627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:38.580351 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.580305 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:38.580559 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.580403 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:38.580559 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.580469 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 21:59:38.580559 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.580509 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 21:59:38.580559 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.580523 2554 projected.go:196] Error preparing data for projected volume kube-api-access-p9lvd for pod openshift-network-diagnostics/network-check-target-8nw5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:38.580794 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.580588 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:38.580794 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.580594 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd podName:efe077f6-5e11-4e9a-bb32-b6f200b4fd76 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:39.580573898 +0000 UTC m=+4.145422461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9lvd" (UniqueName: "kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd") pod "network-check-target-8nw5p" (UID: "efe077f6-5e11-4e9a-bb32-b6f200b4fd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:38.580794 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.580685 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 21:59:39.580666968 +0000 UTC m=+4.145515533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:38.951164 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.951085 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:38.951955 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:38.951229 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:38.981097 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.981041 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" event={"ID":"17e24bb8-7793-491d-b095-af57702b62af","Type":"ContainerStarted","Data":"62f92e2a258a9075d1ec3b9b98b36c69fcd47f2eb9a01016238fc65a4f62388e"} Jul 31 21:59:38.986381 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.986262 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"42f132fc708ccee09417590f734181eb39fe79246e02261db11683b23aab3a15"} Jul 31 21:59:38.988550 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.988482 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cr798" event={"ID":"7ee82f9d-1e38-4eb9-9997-53f53c456c1d","Type":"ContainerStarted","Data":"c59be5574e94ae00b4076349a73b6de18ba20106064593fa2cb2e9864d3912a3"} Jul 31 21:59:38.990204 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.990147 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5rxkg" event={"ID":"d67851f3-1f5a-44f0-838e-0279d191776c","Type":"ContainerStarted","Data":"7308d24c4dab0b3452015890ed1f3eff55ad8b5bbf9bf03eee5b0da3d36937da"} Jul 31 21:59:38.992305 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.992241 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vt8r7" event={"ID":"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7","Type":"ContainerStarted","Data":"a5155d9a57dc4c04110c67f4401812943769446339891630e9e45108e6df1b1d"} Jul 31 21:59:38.995111 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.994816 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" event={"ID":"0789200e-8da2-45b2-be9d-1939083a0505","Type":"ContainerStarted","Data":"67122ff478ea7b25ea8ddce9d429f8198f0c02d002473e3cc469e76d8e58ed18"} Jul 31 21:59:38.998369 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:38.998290 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerStarted","Data":"5ff9a2c1b2eaa0d24d7343471660e5a42d7ec8100d5183f3f3b539b9b2c8e472"} Jul 31 21:59:39.000171 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.000091 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xhptq" event={"ID":"2ed8dd22-e69c-44e5-8184-1694658e7c85","Type":"ContainerStarted","Data":"6c6a1778d9cd55383ef9f11dee366160921573b1bf1e8502f729380c24495a2e"} Jul 31 21:59:39.003106 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.003044 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b8nxl" event={"ID":"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418","Type":"ContainerStarted","Data":"e7e1ccadb6991db9fd226dd49c4157a59adbe4753da33973ade17adb905f035e"} Jul 31 21:59:39.036580 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.036492 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 21:54:38 +0000 UTC" deadline="2028-04-16 19:07:11.24404947 +0000 UTC" Jul 31 21:59:39.036580 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.036526 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14997h7m32.207528591s" Jul 31 21:59:39.490249 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.489645 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:39.490249 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.489811 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:39.490249 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.489881 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:41.489861993 +0000 UTC m=+6.054710557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:39.590527 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.590481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:39.590732 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.590563 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:39.590803 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.590743 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:39.590882 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.590811 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 21:59:41.590790997 +0000 UTC m=+6.155639586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:39.591269 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.591242 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 21:59:39.591269 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.591266 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 21:59:39.591431 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.591280 2554 projected.go:196] Error preparing data for projected volume kube-api-access-p9lvd for pod openshift-network-diagnostics/network-check-target-8nw5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:39.591431 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.591369 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd podName:efe077f6-5e11-4e9a-bb32-b6f200b4fd76 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:41.591353772 +0000 UTC m=+6.156202336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9lvd" (UniqueName: "kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd") pod "network-check-target-8nw5p" (UID: "efe077f6-5e11-4e9a-bb32-b6f200b4fd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:39.953155 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.952807 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:39.953155 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.952941 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:39.953155 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:39.952954 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:39.953155 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:39.953049 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:40.036953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:40.036874 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 21:54:38 +0000 UTC" deadline="2028-03-18 10:04:13.977810319 +0000 UTC" Jul 31 21:59:40.036953 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:40.036920 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14292h4m33.940895499s" Jul 31 21:59:40.950797 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:40.950570 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:40.950797 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:40.950764 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:41.509652 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:41.509608 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:41.510273 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.509811 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:41.510273 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.509874 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:45.509856094 +0000 UTC m=+10.074704656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:41.611173 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:41.611134 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:41.611350 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:41.611206 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:41.611350 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.611341 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:41.611516 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.611406 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 21:59:45.611386168 +0000 UTC m=+10.176234732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:41.611792 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.611766 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 21:59:41.611885 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.611799 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 21:59:41.611885 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.611813 2554 projected.go:196] Error preparing data for projected volume kube-api-access-p9lvd for pod openshift-network-diagnostics/network-check-target-8nw5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:41.611885 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.611870 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd podName:efe077f6-5e11-4e9a-bb32-b6f200b4fd76 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:45.611853686 +0000 UTC m=+10.176702260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9lvd" (UniqueName: "kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd") pod "network-check-target-8nw5p" (UID: "efe077f6-5e11-4e9a-bb32-b6f200b4fd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:41.951697 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:41.951609 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:41.951859 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:41.951631 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:41.951859 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.951783 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:41.952321 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:41.952281 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:42.951415 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:42.951324 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:42.951867 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:42.951467 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:43.950939 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:43.950851 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:43.951135 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:43.950992 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:43.951135 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:43.951075 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:43.951258 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:43.951187 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:44.950559 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:44.950524 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:44.951029 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:44.950671 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:45.543735 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:45.543699 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:45.543911 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.543846 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:45.543990 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.543919 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:53.543900889 +0000 UTC m=+18.108749472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:45.644978 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:45.644935 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:45.645162 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:45.645006 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:45.645162 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.645085 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 21:59:45.645162 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.645102 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:45.645162 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.645107 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 21:59:45.645162 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.645120 2554 projected.go:196] Error preparing data for projected volume kube-api-access-p9lvd for pod openshift-network-diagnostics/network-check-target-8nw5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:45.645162 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.645161 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 21:59:53.645141268 +0000 UTC m=+18.209989851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:45.645485 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.645180 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd podName:efe077f6-5e11-4e9a-bb32-b6f200b4fd76 nodeName:}" failed. No retries permitted until 2026-07-31 21:59:53.645170056 +0000 UTC m=+18.210018615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9lvd" (UniqueName: "kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd") pod "network-check-target-8nw5p" (UID: "efe077f6-5e11-4e9a-bb32-b6f200b4fd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:45.951206 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:45.951135 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:45.951576 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.951251 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:45.951576 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:45.951303 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:45.951576 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:45.951387 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:46.951499 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:46.951389 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:46.951949 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:46.951524 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:47.951226 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:47.951191 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:47.951402 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:47.951347 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:47.951483 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:47.951400 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:47.951536 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:47.951515 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:48.951104 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:48.951078 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:48.951210 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:48.951183 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:49.032140 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:49.031415 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" event={"ID":"01e690b2a67d9f68121efde86b8c3543","Type":"ContainerStarted","Data":"b105d6a5a7023b7faad691709cdf8f44d0cc78eb655c4ac6173d157120fae72c"} Jul 31 21:59:49.034356 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:49.034010 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" event={"ID":"0789200e-8da2-45b2-be9d-1939083a0505","Type":"ContainerStarted","Data":"ca8c1e084704ba9f445270dcdc031dba1d54305ea6af3cd13b231c3b42c1e0e3"} Jul 31 21:59:49.050176 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:49.050128 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-197.ec2.internal" podStartSLOduration=12.050110825 podStartE2EDuration="12.050110825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:49.049341053 +0000 UTC m=+13.614189636" watchObservedRunningTime="2026-07-31 21:59:49.050110825 +0000 UTC m=+13.614959412" Jul 31 21:59:49.069108 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:49.069057 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d7vlw" podStartSLOduration=2.434226491 podStartE2EDuration="13.069039111s" totalImagesPullingTime="10.63481262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:49.068970524 +0000 UTC m=+13.633819106" watchObservedRunningTime="2026-07-31 21:59:49.069039111 +0000 UTC m=+13.633887696" Jul 31 21:59:49.951656 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:49.951626 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:49.951656 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:49.951661 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:49.951871 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:49.951758 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:49.951910 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:49.951876 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:50.037555 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.037515 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cr798" event={"ID":"7ee82f9d-1e38-4eb9-9997-53f53c456c1d","Type":"ContainerStarted","Data":"f2e8b9733a23a8d9a25e46ce816798e4d77cd305d550f315ce91310151cec2f9"} Jul 31 21:59:50.040664 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.039461 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" event={"ID":"a563827a727e1fa7ade2716ea0dbf4d5","Type":"ContainerStarted","Data":"d902c07af36c3371f562f4b63e3a4d106da126d94d087e928ce0f2064627f9d5"} Jul 31 21:59:50.045478 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.045442 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5rxkg" event={"ID":"d67851f3-1f5a-44f0-838e-0279d191776c","Type":"ContainerStarted","Data":"e26cf4e9471f38030a518226525e4d2a3c318a95c184e3ce4ee4f9939bd0933f"} Jul 31 21:59:50.047068 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.047039 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vt8r7" event={"ID":"b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7","Type":"ContainerStarted","Data":"ca3962999414d7d4a15b10c8fa4346366e2279631f5e7acf34163a87efb1a9fb"} Jul 31 21:59:50.048577 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.048545 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerStarted","Data":"dea82e14d9ee9115bb008184d7fe3c3579660c0a28ea1acf84e005b0f9da4602"} Jul 31 21:59:50.050085 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.050061 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xhptq" event={"ID":"2ed8dd22-e69c-44e5-8184-1694658e7c85","Type":"ContainerStarted","Data":"0606affa64949bd78fa8452b26abcf4a1226bfda5979ddbd541ea0dd97473beb"} Jul 31 21:59:50.051735 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.051667 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" event={"ID":"17e24bb8-7793-491d-b095-af57702b62af","Type":"ContainerStarted","Data":"a15bf117de191e12c5743899bca27480ae2d93f9b58ee64b666d07b214805e22"} Jul 31 21:59:50.060823 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.060780 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cr798" podStartSLOduration=3.578253945 podStartE2EDuration="14.060765014s" totalImagesPullingTime="10.482511069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:50.05999449 +0000 UTC m=+14.624843075" watchObservedRunningTime="2026-07-31 21:59:50.060765014 +0000 UTC m=+14.625613596" Jul 31 21:59:50.084141 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.084100 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vt8r7" podStartSLOduration=3.635763255 podStartE2EDuration="14.084086708s" totalImagesPullingTime="10.448323453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:50.083553961 +0000 UTC m=+14.648402546" watchObservedRunningTime="2026-07-31 21:59:50.084086708 +0000 UTC m=+14.648935289" Jul 31 21:59:50.107146 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.107103 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xhptq" podStartSLOduration=3.614767923 podStartE2EDuration="14.107090621s" totalImagesPullingTime="10.492322698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:50.107062103 +0000 UTC m=+14.671910688" watchObservedRunningTime="2026-07-31 21:59:50.107090621 +0000 UTC m=+14.671939204" Jul 31 21:59:50.154050 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.154002 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5rxkg" podStartSLOduration=3.714576078 podStartE2EDuration="14.153986128s" totalImagesPullingTime="10.43941005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:50.132335735 +0000 UTC m=+14.697184312" watchObservedRunningTime="2026-07-31 21:59:50.153986128 +0000 UTC m=+14.718834709" Jul 31 21:59:50.950799 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:50.950774 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:50.950992 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:50.950880 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:51.951582 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:51.951545 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:51.952007 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:51.951680 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:51.952007 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:51.951748 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:51.952007 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:51.951827 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:52.950673 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:52.950635 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:52.950874 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:52.950775 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:53.059738 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.059650 2554 generic.go:356] "Generic (PLEG): container finished" podID="a563827a727e1fa7ade2716ea0dbf4d5" containerID="d902c07af36c3371f562f4b63e3a4d106da126d94d087e928ce0f2064627f9d5" exitCode=0 Jul 31 21:59:53.059738 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.059715 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" event={"ID":"a563827a727e1fa7ade2716ea0dbf4d5","Type":"ContainerDied","Data":"d902c07af36c3371f562f4b63e3a4d106da126d94d087e928ce0f2064627f9d5"} Jul 31 21:59:53.061447 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.061425 2554 generic.go:356] "Generic (PLEG): container finished" podID="790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0" containerID="dea82e14d9ee9115bb008184d7fe3c3579660c0a28ea1acf84e005b0f9da4602" exitCode=0 Jul 31 21:59:53.061547 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.061462 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerDied","Data":"dea82e14d9ee9115bb008184d7fe3c3579660c0a28ea1acf84e005b0f9da4602"} Jul 31 21:59:53.604659 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.604627 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:53.604851 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.604755 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:53.604851 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.604817 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:09.604799396 +0000 UTC m=+34.169647955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 21:59:53.705234 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.705202 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:53.705426 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.705275 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:53.705426 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.705359 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:53.705426 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.705383 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 31 21:59:53.705426 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.705402 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 31 21:59:53.705426 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.705414 2554 projected.go:196] Error preparing data for projected volume kube-api-access-p9lvd for pod openshift-network-diagnostics/network-check-target-8nw5p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:53.705426 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.705423 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:09.705408602 +0000 UTC m=+34.270257162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 21:59:53.705753 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.705451 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd podName:efe077f6-5e11-4e9a-bb32-b6f200b4fd76 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:09.705441344 +0000 UTC m=+34.270289908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9lvd" (UniqueName: "kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd") pod "network-check-target-8nw5p" (UID: "efe077f6-5e11-4e9a-bb32-b6f200b4fd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 31 21:59:53.950763 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.950683 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:53.950922 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.950815 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:53.950922 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:53.950683 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:53.951043 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:53.950975 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:54.086396 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:54.086349 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:54.087094 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:54.087071 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:54.951450 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:54.951418 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:54.951635 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:54.951532 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:55.065304 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:55.065270 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:55.065888 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:55.065866 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5rxkg" Jul 31 21:59:55.951851 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:55.951814 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:55.952294 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:55.951896 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:55.952294 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:55.951907 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:55.952294 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:55.952020 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:56.950854 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:56.950819 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:56.951041 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:56.950937 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:57.469110 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:57.468927 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 31 21:59:57.924097 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:57.923831 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T21:59:57.469109301Z","UUID":"797ac51f-8300-4794-89e3-35fea5f82676","Handler":null,"Name":"","Endpoint":""} Jul 31 21:59:57.925549 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:57.925527 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 31 21:59:57.925681 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:57.925573 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 31 21:59:57.950612 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:57.950563 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:57.950747 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:57.950720 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 21:59:57.950747 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:57.950731 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:57.950832 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:57.950816 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:58.073465 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.073380 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" event={"ID":"a563827a727e1fa7ade2716ea0dbf4d5","Type":"ContainerStarted","Data":"6e4a5e353a4ab80e278b5e7f3bd4788a616da910ec499c4290812800d5475769"} Jul 31 21:59:58.075973 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.075945 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b8nxl" event={"ID":"6d90bcf9-d9dd-4c3f-a5b8-a5e0de58c418","Type":"ContainerStarted","Data":"d1add4c567cdb679d9dddccbcee442b0d78b51eaaf4e2453c33356526a684402"} Jul 31 21:59:58.078015 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.077987 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" event={"ID":"17e24bb8-7793-491d-b095-af57702b62af","Type":"ContainerStarted","Data":"5116c2e073289fb71c43a1f586e6b10b6b01fc8e9008ae1dfaa8be4850f9f787"} Jul 31 21:59:58.081642 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.081623 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"4f84f3771741a0d9d3534508799d14a3cd40c179e50105e417b2bcba201ab2ed"} Jul 31 21:59:58.081735 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.081692 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"7561af3df1b7b8e365562e675c5f22e9d03bf9030d6c5f7be917b84de85ed44c"} Jul 31 21:59:58.081735 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.081726 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"39731ff73fad5dd3bbd071c726ef74f9c8392d7bc9c7d6db7a3cf17a845a086a"} Jul 31 21:59:58.081810 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.081757 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"ba64e30f1dfcfe8d95e734613701a20a597c4e24f3485751cc5155d84332d40d"} Jul 31 21:59:58.081810 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.081794 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"8e6c0d03b58dd849b9e94814014aee911507f144df8acb349c854a404f6f1892"} Jul 31 21:59:58.081871 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.081822 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"9dc784681046242f0a36509b392d40433fc03c755bbd439cb96daf644941fd1d"} Jul 31 21:59:58.092493 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.092413 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-197.ec2.internal" podStartSLOduration=21.092397685 podStartE2EDuration="21.092397685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:58.09161509 +0000 UTC m=+22.656463671" watchObservedRunningTime="2026-07-31 21:59:58.092397685 +0000 UTC m=+22.657246271" Jul 31 21:59:58.110524 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.110479 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b8nxl" podStartSLOduration=3.060002118 podStartE2EDuration="22.110462081s" totalImagesPullingTime="19.050459963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:58.109758707 +0000 UTC m=+22.674607291" watchObservedRunningTime="2026-07-31 21:59:58.110462081 +0000 UTC m=+22.675310664" Jul 31 21:59:58.951264 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:58.951226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 21:59:58.951705 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:58.951364 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 21:59:59.086561 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:59.086450 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" event={"ID":"17e24bb8-7793-491d-b095-af57702b62af","Type":"ContainerStarted","Data":"ef553756680f943f07ca73799d047e126c4b050ddbc421dfa6cfc90ddfe34667"} Jul 31 21:59:59.106206 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:59.106159 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-42cpf" podStartSLOduration=3.116610317 podStartE2EDuration="23.106143288s" totalImagesPullingTime="19.989532971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 21:59:59.106016535 +0000 UTC m=+23.670865119" watchObservedRunningTime="2026-07-31 21:59:59.106143288 +0000 UTC m=+23.670991870" Jul 31 21:59:59.954740 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:59.954707 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 21:59:59.955351 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:59.954817 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 21:59:59.955351 ip-10-0-140-197 kubenswrapper[2554]: I0731 21:59:59.954869 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 21:59:59.955351 ip-10-0-140-197 kubenswrapper[2554]: E0731 21:59:59.954990 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 22:00:00.951190 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:00.951153 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:00.951366 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:00.951267 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 22:00:01.950960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:01.950883 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:01.951485 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:01.950893 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:01.951485 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:01.951018 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 22:00:01.951485 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:01.951054 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 22:00:02.094077 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:02.094044 2554 generic.go:356] "Generic (PLEG): container finished" podID="790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0" containerID="299aced4d19c2cccb3e63c7f0494cc5df1c1cca0ef23afd1acde6222feb77581" exitCode=0 Jul 31 22:00:02.094259 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:02.094122 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerDied","Data":"299aced4d19c2cccb3e63c7f0494cc5df1c1cca0ef23afd1acde6222feb77581"} Jul 31 22:00:02.097510 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:02.097485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"7446963bf6f10ec0e4d26474339d5fff1b706a1307a4aeebfa7fccbca602c886"} Jul 31 22:00:02.951407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:02.951380 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:02.951791 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:02.951474 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 22:00:03.951615 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:03.951416 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:03.952121 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:03.951449 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:03.952121 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:03.951728 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 22:00:03.952121 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:03.951772 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 22:00:04.105084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.105005 2554 generic.go:356] "Generic (PLEG): container finished" podID="790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0" containerID="47903d5ef567a042d19b5d5a346d1c177efb1ac00636c3a653d4d5ef0a82308c" exitCode=0 Jul 31 22:00:04.105226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.105080 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerDied","Data":"47903d5ef567a042d19b5d5a346d1c177efb1ac00636c3a653d4d5ef0a82308c"} Jul 31 22:00:04.109032 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.109005 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" event={"ID":"5166b1fa-37db-4e80-8424-5c0f4ece91a3","Type":"ContainerStarted","Data":"e03a3fc88421175c862cbb781c05856275886444ba6088560d3b87ae8ff32245"} Jul 31 22:00:04.109269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.109250 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 22:00:04.109363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.109313 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 22:00:04.109363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.109362 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 22:00:04.124865 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.124839 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 22:00:04.124985 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.124952 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 22:00:04.951244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:04.951208 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:04.951394 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:04.951348 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 22:00:05.505441 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.505060 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" podStartSLOduration=10.415418245 podStartE2EDuration="29.505036768s" totalImagesPullingTime="19.089618523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:04.152937819 +0000 UTC m=+28.717786401" watchObservedRunningTime="2026-07-31 22:00:05.505036768 +0000 UTC m=+30.069885347" Jul 31 22:00:05.506037 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.505988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qqgnd"] Jul 31 22:00:05.506181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.506152 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:05.506333 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:05.506305 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 22:00:05.508825 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.508804 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nw5p"] Jul 31 22:00:05.508928 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.508899 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:05.508999 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:05.508980 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 22:00:05.511001 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.510978 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hkdcf"] Jul 31 22:00:05.511086 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:05.511072 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:05.511220 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:05.511195 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 22:00:06.115762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:06.115732 2554 generic.go:356] "Generic (PLEG): container finished" podID="790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0" containerID="dc336f15509e2d8af2bf9e4fbd9db2e6f91130ab60b48d35151f7b1b038e066e" exitCode=0 Jul 31 22:00:06.115930 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:06.115807 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerDied","Data":"dc336f15509e2d8af2bf9e4fbd9db2e6f91130ab60b48d35151f7b1b038e066e"} Jul 31 22:00:06.951407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:06.951232 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:06.951798 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:06.951481 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 22:00:07.951654 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:07.951615 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:07.952099 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:07.951762 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqgnd" podUID="d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e" Jul 31 22:00:07.952099 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:07.951797 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:07.952099 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:07.951941 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hkdcf" podUID="a473fec7-53b1-4440-b7cf-5cdaf810c897" Jul 31 22:00:08.951283 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:08.951256 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:08.951442 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:08.951416 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nw5p" podUID="efe077f6-5e11-4e9a-bb32-b6f200b4fd76" Jul 31 22:00:09.248655 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.248627 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-197.ec2.internal" event="NodeReady" Jul 31 22:00:09.249098 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.248777 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 31 22:00:09.303472 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.303428 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54bbc9d58-vq54h"] Jul 31 22:00:09.330016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.329916 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7"] Jul 31 22:00:09.330179 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.330040 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.334068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.333937 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 31 22:00:09.334270 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.334250 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 31 22:00:09.334873 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.334826 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 31 22:00:09.334873 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.334838 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-55g2j\"" Jul 31 22:00:09.341185 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.341146 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 31 22:00:09.346028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.345998 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522"] Jul 31 22:00:09.346145 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.346130 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:09.349864 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.349844 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.352932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.351180 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 31 22:00:09.352932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.351308 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k8l2r\"" Jul 31 22:00:09.355225 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.355207 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.366107 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.366063 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-bf69l"] Jul 31 22:00:09.366231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.366192 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" Jul 31 22:00:09.368881 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.368845 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.369001 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.368980 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2qq78\"" Jul 31 22:00:09.369251 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.369236 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.388130 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.388104 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7"] Jul 31 22:00:09.388264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.388147 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-bf69l"] Jul 31 22:00:09.388264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.388165 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54bbc9d58-vq54h"] Jul 31 22:00:09.388264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.388180 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522"] Jul 31 22:00:09.388264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.388204 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99"] Jul 31 22:00:09.388264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.388255 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.398533 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.398509 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-j667c\"" Jul 31 22:00:09.399308 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.399287 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 31 22:00:09.400065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.400045 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.401969 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.401950 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 31 22:00:09.406324 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.406305 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.411889 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.411860 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:09.412311 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.412293 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-644f8b4f6-vsfsr"] Jul 31 22:00:09.412438 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.412415 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.416612 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.416583 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.416714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.416682 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.417120 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.417104 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 31 22:00:09.417228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.417211 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 31 22:00:09.421481 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.421464 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 31 22:00:09.421730 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.421706 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-v7c68\"" Jul 31 22:00:09.422966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.422938 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.422974 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-installation-pull-secrets\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.423034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-image-registry-private-configuration\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.423059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-trusted-ca\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.423096 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdhzw\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-kube-api-access-kdhzw\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423183 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.423150 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-certificates\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.423195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-ca-trust-extracted\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.423226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.423219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-bound-sa-token\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.430795 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.430771 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m"] Jul 31 22:00:09.430954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.430938 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.433818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.433801 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 31 22:00:09.436684 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.436665 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 31 22:00:09.436684 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.436684 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 31 22:00:09.438246 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.438220 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 31 22:00:09.438246 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.438221 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tjjh6\"" Jul 31 22:00:09.456366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.456344 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44"] Jul 31 22:00:09.456545 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.456476 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.459874 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.459857 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.460043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.460030 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.460100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.460066 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 31 22:00:09.460172 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.460157 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mlxpz\"" Jul 31 22:00:09.461742 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.461728 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.462773 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.462751 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.464538 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.464520 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 31 22:00:09.476639 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.476620 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-m67tr"] Jul 31 22:00:09.476735 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.476720 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.494654 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.494633 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4"] Jul 31 22:00:09.494766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.494753 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.504754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.504730 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 31 22:00:09.504754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.504735 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 31 22:00:09.504911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.504820 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 31 22:00:09.504911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.504832 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-k58pv\"" Jul 31 22:00:09.504911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.504861 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7gs5d\"" Jul 31 22:00:09.504911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.504841 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.505931 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.505475 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.505931 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.505761 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.506563 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.506544 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 31 22:00:09.508944 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.508788 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.510990 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.510969 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 31 22:00:09.512712 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512696 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44"] Jul 31 22:00:09.512789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512722 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m"] Jul 31 22:00:09.512789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512736 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99"] Jul 31 22:00:09.512789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512750 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-m67tr"] Jul 31 22:00:09.512789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512762 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-644f8b4f6-vsfsr"] Jul 31 22:00:09.512789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512778 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4"] Jul 31 22:00:09.512960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.512810 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" Jul 31 22:00:09.515759 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.515741 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.515855 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.515771 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rwn2d\"" Jul 31 22:00:09.516429 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.516413 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.523748 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523723 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-serving-cert\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.523833 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523766 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.523889 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-image-registry-private-configuration\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.523952 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523888 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:09.523952 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523915 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh6b9\" (UniqueName: \"kubernetes.io/projected/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-kube-api-access-qh6b9\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.523952 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-trusted-ca\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.524100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.523970 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-stats-auth\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.524100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524016 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-trusted-ca\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524071 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/22deacc6-c170-4286-9dab-f20d65fe111b-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.524254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524102 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdhzw\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-kube-api-access-kdhzw\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524129 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.524254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524160 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-certificates\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524182 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-config\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.524254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524218 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhs88\" (UniqueName: \"kubernetes.io/projected/1c31e4e4-71ec-4ce6-9b4f-cc004da9f007-kube-api-access-jhs88\") pod \"volume-data-source-validator-5b8776669-dk522\" (UID: \"1c31e4e4-71ec-4ce6-9b4f-cc004da9f007\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" Jul 31 22:00:09.524254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524245 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-ca-trust-extracted\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524270 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-bound-sa-token\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524295 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-default-certificate\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524318 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524344 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22w6m\" (UniqueName: \"kubernetes.io/projected/07c72711-0eaf-4064-b343-3572c0783bf0-kube-api-access-22w6m\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524370 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9r7p\" (UniqueName: \"kubernetes.io/projected/22deacc6-c170-4286-9dab-f20d65fe111b-kube-api-access-q9r7p\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524466 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-installation-pull-secrets\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.524519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524490 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbc5b\" (UniqueName: \"kubernetes.io/projected/5099f668-6faf-4fba-a215-af7fed8091cd-kube-api-access-jbc5b\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:09.524889 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.524659 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 31 22:00:09.524889 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.524673 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbc9d58-vq54h: secret "image-registry-tls" not found Jul 31 22:00:09.524889 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.524724 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls podName:5ce1f101-780d-4c40-b7ce-6d9d7844efc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:10.024706254 +0000 UTC m=+34.589554818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls") pod "image-registry-54bbc9d58-vq54h" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8") : secret "image-registry-tls" not found Jul 31 22:00:09.525250 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524910 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-ca-trust-extracted\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.525250 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.524952 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-trusted-ca\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.525250 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.525036 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-certificates\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.528950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.528928 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-installation-pull-secrets\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.528950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.528940 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-image-registry-private-configuration\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.549027 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.548997 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdhzw\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-kube-api-access-kdhzw\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.552257 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.552230 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-bound-sa-token\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:09.625097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625063 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.625280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625104 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9r7p\" (UniqueName: \"kubernetes.io/projected/22deacc6-c170-4286-9dab-f20d65fe111b-kube-api-access-q9r7p\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.625280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625148 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8g2s\" (UniqueName: \"kubernetes.io/projected/79ae70c5-c782-4984-99b6-77536ebc3f80-kube-api-access-h8g2s\") pod \"network-check-source-65bdbbff45-br2c4\" (UID: \"79ae70c5-c782-4984-99b6-77536ebc3f80\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" Jul 31 22:00:09.625280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625186 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b3d2cae-4201-4a92-abab-c15ff7865fbf-snapshots\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.625280 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.625256 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:10.125234202 +0000 UTC m=+34.690082764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : configmap references non-existent config key: service-ca.crt Jul 31 22:00:09.625495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625319 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:09.625495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625361 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-trusted-ca\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.625495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625386 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-stats-auth\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.625495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625412 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a19c41c2-668e-495f-aa3f-e9493d1e94c7-config\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.625495 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.625434 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.625501 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls podName:5099f668-6faf-4fba-a215-af7fed8091cd nodeName:}" failed. No retries permitted until 2026-07-31 22:00:10.125482757 +0000 UTC m=+34.690331332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-8wzr7" (UID: "5099f668-6faf-4fba-a215-af7fed8091cd") : secret "samples-operator-tls" not found Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625438 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b3d2cae-4201-4a92-abab-c15ff7865fbf-serving-cert\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625582 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-config\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625651 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b3d2cae-4201-4a92-abab-c15ff7865fbf-service-ca-bundle\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625683 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhs88\" (UniqueName: \"kubernetes.io/projected/1c31e4e4-71ec-4ce6-9b4f-cc004da9f007-kube-api-access-jhs88\") pod \"volume-data-source-validator-5b8776669-dk522\" (UID: \"1c31e4e4-71ec-4ce6-9b4f-cc004da9f007\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625710 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfj6n\" (UniqueName: \"kubernetes.io/projected/502ad9e6-5e67-417b-9efb-6e1625647fd1-kube-api-access-xfj6n\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625738 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-default-certificate\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.625779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625763 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22w6m\" (UniqueName: \"kubernetes.io/projected/07c72711-0eaf-4064-b343-3572c0783bf0-kube-api-access-22w6m\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.626141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625808 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b3d2cae-4201-4a92-abab-c15ff7865fbf-tmp\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.626141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.625833 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx5tk\" (UniqueName: \"kubernetes.io/projected/5b3d2cae-4201-4a92-abab-c15ff7865fbf-kube-api-access-qx5tk\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.626141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626004 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.626141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626066 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbc5b\" (UniqueName: \"kubernetes.io/projected/5099f668-6faf-4fba-a215-af7fed8091cd-kube-api-access-jbc5b\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:09.626141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626098 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a19c41c2-668e-495f-aa3f-e9493d1e94c7-serving-cert\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.626387 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.626169 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 22:00:09.626387 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.626222 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:10.126206995 +0000 UTC m=+34.691055561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : secret "router-metrics-certs-default" not found Jul 31 22:00:09.626485 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626447 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-serving-cert\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.626485 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626457 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-trusted-ca\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.626638 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626491 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:09.626638 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626498 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-config\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.626638 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626571 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6b9\" (UniqueName: \"kubernetes.io/projected/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-kube-api-access-qh6b9\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.626638 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626630 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/502ad9e6-5e67-417b-9efb-6e1625647fd1-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.626875 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626832 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/22deacc6-c170-4286-9dab-f20d65fe111b-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.626938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626878 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b3d2cae-4201-4a92-abab-c15ff7865fbf-trusted-ca-bundle\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.626938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626908 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m56d6\" (UniqueName: \"kubernetes.io/projected/a19c41c2-668e-495f-aa3f-e9493d1e94c7-kube-api-access-m56d6\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.627036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626942 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.627036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.626981 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/502ad9e6-5e67-417b-9efb-6e1625647fd1-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.627133 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.627094 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:09.627180 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.627144 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls podName:22deacc6-c170-4286-9dab-f20d65fe111b nodeName:}" failed. No retries permitted until 2026-07-31 22:00:10.12712931 +0000 UTC m=+34.691977874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-h4d99" (UID: "22deacc6-c170-4286-9dab-f20d65fe111b") : secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:09.627301 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.627283 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 31 22:00:09.627361 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.627330 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret podName:a473fec7-53b1-4440-b7cf-5cdaf810c897 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.627315445 +0000 UTC m=+66.192164005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret") pod "global-pull-secret-syncer-hkdcf" (UID: "a473fec7-53b1-4440-b7cf-5cdaf810c897") : object "kube-system"/"original-pull-secret" not registered Jul 31 22:00:09.627567 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.627546 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/22deacc6-c170-4286-9dab-f20d65fe111b-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.628616 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.628528 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-stats-auth\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.629242 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.629214 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-serving-cert\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.629678 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.629652 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-default-certificate\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.656066 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.656036 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6b9\" (UniqueName: \"kubernetes.io/projected/730d0f93-98f3-42a8-9ad2-a153a2cd71f9-kube-api-access-qh6b9\") pod \"console-operator-5ccd6b445f-bf69l\" (UID: \"730d0f93-98f3-42a8-9ad2-a153a2cd71f9\") " pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.656066 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.656053 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9r7p\" (UniqueName: \"kubernetes.io/projected/22deacc6-c170-4286-9dab-f20d65fe111b-kube-api-access-q9r7p\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:09.692068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.692034 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbc5b\" (UniqueName: \"kubernetes.io/projected/5099f668-6faf-4fba-a215-af7fed8091cd-kube-api-access-jbc5b\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:09.692217 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.692122 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhs88\" (UniqueName: \"kubernetes.io/projected/1c31e4e4-71ec-4ce6-9b4f-cc004da9f007-kube-api-access-jhs88\") pod \"volume-data-source-validator-5b8776669-dk522\" (UID: \"1c31e4e4-71ec-4ce6-9b4f-cc004da9f007\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" Jul 31 22:00:09.692538 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.692514 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22w6m\" (UniqueName: \"kubernetes.io/projected/07c72711-0eaf-4064-b343-3572c0783bf0-kube-api-access-22w6m\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:09.697943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.697910 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:09.705561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.705540 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4466l"] Jul 31 22:00:09.728300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728270 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b3d2cae-4201-4a92-abab-c15ff7865fbf-service-ca-bundle\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.728300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728311 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfj6n\" (UniqueName: \"kubernetes.io/projected/502ad9e6-5e67-417b-9efb-6e1625647fd1-kube-api-access-xfj6n\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.728517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728349 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:09.728517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728374 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b3d2cae-4201-4a92-abab-c15ff7865fbf-tmp\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.728517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728392 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx5tk\" (UniqueName: \"kubernetes.io/projected/5b3d2cae-4201-4a92-abab-c15ff7865fbf-kube-api-access-qx5tk\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.728517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728425 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a19c41c2-668e-495f-aa3f-e9493d1e94c7-serving-cert\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.728517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728476 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/502ad9e6-5e67-417b-9efb-6e1625647fd1-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.728517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728512 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b3d2cae-4201-4a92-abab-c15ff7865fbf-trusted-ca-bundle\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.728534 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.728618 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs podName:d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.72858318 +0000 UTC m=+66.293431753 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs") pod "network-metrics-daemon-qqgnd" (UID: "d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728537 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m56d6\" (UniqueName: \"kubernetes.io/projected/a19c41c2-668e-495f-aa3f-e9493d1e94c7-kube-api-access-m56d6\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728685 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/502ad9e6-5e67-417b-9efb-6e1625647fd1-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728734 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8g2s\" (UniqueName: \"kubernetes.io/projected/79ae70c5-c782-4984-99b6-77536ebc3f80-kube-api-access-h8g2s\") pod \"network-check-source-65bdbbff45-br2c4\" (UID: \"79ae70c5-c782-4984-99b6-77536ebc3f80\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728767 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728796 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b3d2cae-4201-4a92-abab-c15ff7865fbf-snapshots\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728837 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a19c41c2-668e-495f-aa3f-e9493d1e94c7-config\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728869 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b3d2cae-4201-4a92-abab-c15ff7865fbf-serving-cert\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.729088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.729063 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b3d2cae-4201-4a92-abab-c15ff7865fbf-service-ca-bundle\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.729697 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.729486 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/502ad9e6-5e67-417b-9efb-6e1625647fd1-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.729697 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.728798 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b3d2cae-4201-4a92-abab-c15ff7865fbf-tmp\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.729793 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.729750 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b3d2cae-4201-4a92-abab-c15ff7865fbf-trusted-ca-bundle\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.730055 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.730001 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b3d2cae-4201-4a92-abab-c15ff7865fbf-snapshots\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.730147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.730064 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a19c41c2-668e-495f-aa3f-e9493d1e94c7-config\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.731629 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.731586 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/502ad9e6-5e67-417b-9efb-6e1625647fd1-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.731886 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.731863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b3d2cae-4201-4a92-abab-c15ff7865fbf-serving-cert\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.731995 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.731971 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a19c41c2-668e-495f-aa3f-e9493d1e94c7-serving-cert\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.732565 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.732547 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lvd\" (UniqueName: \"kubernetes.io/projected/efe077f6-5e11-4e9a-bb32-b6f200b4fd76-kube-api-access-p9lvd\") pod \"network-check-target-8nw5p\" (UID: \"efe077f6-5e11-4e9a-bb32-b6f200b4fd76\") " pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:09.737436 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.737411 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6cj9w"] Jul 31 22:00:09.737571 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.737553 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.742530 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.742497 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 31 22:00:09.742714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.742696 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 31 22:00:09.742801 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.742748 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jdkvx\"" Jul 31 22:00:09.768222 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.768129 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4466l"] Jul 31 22:00:09.768222 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.768181 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6cj9w"] Jul 31 22:00:09.768485 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.768279 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:09.772364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.772122 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 31 22:00:09.772364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.772189 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 31 22:00:09.772364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.772300 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w8lz5\"" Jul 31 22:00:09.772646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.772417 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 31 22:00:09.775736 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.775678 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfj6n\" (UniqueName: \"kubernetes.io/projected/502ad9e6-5e67-417b-9efb-6e1625647fd1-kube-api-access-xfj6n\") pod \"kube-storage-version-migrator-operator-7878f4b88c-6lb44\" (UID: \"502ad9e6-5e67-417b-9efb-6e1625647fd1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.776681 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.776622 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx5tk\" (UniqueName: \"kubernetes.io/projected/5b3d2cae-4201-4a92-abab-c15ff7865fbf-kube-api-access-qx5tk\") pod \"insights-operator-77696685b-m67tr\" (UID: \"5b3d2cae-4201-4a92-abab-c15ff7865fbf\") " pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.784787 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.784748 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m56d6\" (UniqueName: \"kubernetes.io/projected/a19c41c2-668e-495f-aa3f-e9493d1e94c7-kube-api-access-m56d6\") pod \"service-ca-operator-784f97bcf6-8g46m\" (UID: \"a19c41c2-668e-495f-aa3f-e9493d1e94c7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:09.785654 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.785633 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" Jul 31 22:00:09.795216 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.795195 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8g2s\" (UniqueName: \"kubernetes.io/projected/79ae70c5-c782-4984-99b6-77536ebc3f80-kube-api-access-h8g2s\") pod \"network-check-source-65bdbbff45-br2c4\" (UID: \"79ae70c5-c782-4984-99b6-77536ebc3f80\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" Jul 31 22:00:09.802678 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.802657 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-m67tr" Jul 31 22:00:09.833631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.829344 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-tmp-dir\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.833631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.829447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjl8\" (UniqueName: \"kubernetes.io/projected/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-kube-api-access-qzjl8\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.833631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.829479 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.833631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.829557 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-config-volume\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.833631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.829796 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" Jul 31 22:00:09.843489 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.842779 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:09.933849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.933487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjl8\" (UniqueName: \"kubernetes.io/projected/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-kube-api-access-qzjl8\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.933849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.933785 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.933970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.933887 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-config-volume\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.933970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.933927 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22807a73-4c34-4adc-9993-5b3019dbd876-cert\") pod \"ingress-canary-6cj9w\" (UID: \"22807a73-4c34-4adc-9993-5b3019dbd876\") " pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:09.933970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.933958 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-tmp-dir\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.934105 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.934020 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prwzm\" (UniqueName: \"kubernetes.io/projected/22807a73-4c34-4adc-9993-5b3019dbd876-kube-api-access-prwzm\") pod \"ingress-canary-6cj9w\" (UID: \"22807a73-4c34-4adc-9993-5b3019dbd876\") " pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:09.935666 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.934511 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 22:00:09.935666 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:09.934576 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls podName:f4f6cbf1-c75d-4c44-b103-0cac8cb1480e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:10.434556751 +0000 UTC m=+34.999405319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls") pod "dns-default-4466l" (UID: "f4f6cbf1-c75d-4c44-b103-0cac8cb1480e") : secret "dns-default-metrics-tls" not found Jul 31 22:00:09.935666 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.935201 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-config-volume\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.935666 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.935453 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-tmp-dir\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.951747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.951721 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:09.952719 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.952701 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:09.956561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.956104 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjl8\" (UniqueName: \"kubernetes.io/projected/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-kube-api-access-qzjl8\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:09.956561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.956110 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sb6cn\"" Jul 31 22:00:09.956561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.956206 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 22:00:09.956895 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.956874 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 22:00:09.976909 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:09.976869 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" Jul 31 22:00:10.000827 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.000796 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-bf69l"] Jul 31 22:00:10.013871 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.013844 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:10.035228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.035124 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22807a73-4c34-4adc-9993-5b3019dbd876-cert\") pod \"ingress-canary-6cj9w\" (UID: \"22807a73-4c34-4adc-9993-5b3019dbd876\") " pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:10.035228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.035191 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prwzm\" (UniqueName: \"kubernetes.io/projected/22807a73-4c34-4adc-9993-5b3019dbd876-kube-api-access-prwzm\") pod \"ingress-canary-6cj9w\" (UID: \"22807a73-4c34-4adc-9993-5b3019dbd876\") " pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:10.035390 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.035238 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:10.035440 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.035425 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 31 22:00:10.035492 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.035444 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbc9d58-vq54h: secret "image-registry-tls" not found Jul 31 22:00:10.035543 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.035516 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls podName:5ce1f101-780d-4c40-b7ce-6d9d7844efc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:11.035495419 +0000 UTC m=+35.600343995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls") pod "image-registry-54bbc9d58-vq54h" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8") : secret "image-registry-tls" not found Jul 31 22:00:10.038221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.038198 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22807a73-4c34-4adc-9993-5b3019dbd876-cert\") pod \"ingress-canary-6cj9w\" (UID: \"22807a73-4c34-4adc-9993-5b3019dbd876\") " pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:10.053490 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.053458 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4"] Jul 31 22:00:10.055252 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.055229 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-m67tr"] Jul 31 22:00:10.065249 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.065225 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44"] Jul 31 22:00:10.065348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.065335 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" Jul 31 22:00:10.065733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.065712 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prwzm\" (UniqueName: \"kubernetes.io/projected/22807a73-4c34-4adc-9993-5b3019dbd876-kube-api-access-prwzm\") pod \"ingress-canary-6cj9w\" (UID: \"22807a73-4c34-4adc-9993-5b3019dbd876\") " pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:10.081257 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.081239 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6cj9w" Jul 31 22:00:10.136227 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.136159 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:10.136383 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136291 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 22:00:10.136383 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136354 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:11.136334754 +0000 UTC m=+35.701183313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : secret "router-metrics-certs-default" not found Jul 31 22:00:10.136504 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.136412 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:10.136504 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.136452 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:10.136504 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.136496 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:10.136678 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136545 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:10.136678 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136589 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 22:00:10.136678 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136616 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:11.136584833 +0000 UTC m=+35.701433405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : configmap references non-existent config key: service-ca.crt Jul 31 22:00:10.136678 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136637 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls podName:22deacc6-c170-4286-9dab-f20d65fe111b nodeName:}" failed. No retries permitted until 2026-07-31 22:00:11.136627292 +0000 UTC m=+35.701475864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-h4d99" (UID: "22deacc6-c170-4286-9dab-f20d65fe111b") : secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:10.136678 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.136657 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls podName:5099f668-6faf-4fba-a215-af7fed8091cd nodeName:}" failed. No retries permitted until 2026-07-31 22:00:11.136643002 +0000 UTC m=+35.701491562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-8wzr7" (UID: "5099f668-6faf-4fba-a215-af7fed8091cd") : secret "samples-operator-tls" not found Jul 31 22:00:10.368838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.368753 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:10.439649 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.439614 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:10.439863 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.439784 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 22:00:10.439863 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:10.439861 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls podName:f4f6cbf1-c75d-4c44-b103-0cac8cb1480e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:11.439840395 +0000 UTC m=+36.004688957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls") pod "dns-default-4466l" (UID: "f4f6cbf1-c75d-4c44-b103-0cac8cb1480e") : secret "dns-default-metrics-tls" not found Jul 31 22:00:10.473244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.473209 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:10.772858 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.772754 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:10.834368 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.834331 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:10.951356 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.951326 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:10.954220 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.954195 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nslhh\"" Jul 31 22:00:10.960269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:10.960252 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:11.044067 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.043991 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:11.044215 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.044178 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 31 22:00:11.044215 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.044202 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbc9d58-vq54h: secret "image-registry-tls" not found Jul 31 22:00:11.044302 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.044270 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls podName:5ce1f101-780d-4c40-b7ce-6d9d7844efc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:13.044247311 +0000 UTC m=+37.609095873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls") pod "image-registry-54bbc9d58-vq54h" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8") : secret "image-registry-tls" not found Jul 31 22:00:11.144549 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.144511 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:11.144766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.144564 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:11.144766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.144640 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:11.144766 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.144678 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:11.144766 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.144744 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls podName:22deacc6-c170-4286-9dab-f20d65fe111b nodeName:}" failed. No retries permitted until 2026-07-31 22:00:13.144725439 +0000 UTC m=+37.709574001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-h4d99" (UID: "22deacc6-c170-4286-9dab-f20d65fe111b") : secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:11.144766 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.144764 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:13.144754256 +0000 UTC m=+37.709602820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : configmap references non-existent config key: service-ca.crt Jul 31 22:00:11.144766 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.144764 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 22:00:11.145070 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.144810 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls podName:5099f668-6faf-4fba-a215-af7fed8091cd nodeName:}" failed. No retries permitted until 2026-07-31 22:00:13.144795345 +0000 UTC m=+37.709643912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-8wzr7" (UID: "5099f668-6faf-4fba-a215-af7fed8091cd") : secret "samples-operator-tls" not found Jul 31 22:00:11.145070 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.144851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:11.145070 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.144968 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 22:00:11.145070 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.145006 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:13.144995479 +0000 UTC m=+37.709844042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : secret "router-metrics-certs-default" not found Jul 31 22:00:11.448138 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.448061 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:11.448472 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.448208 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 22:00:11.448472 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:11.448271 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls podName:f4f6cbf1-c75d-4c44-b103-0cac8cb1480e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:13.448257279 +0000 UTC m=+38.013105840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls") pod "dns-default-4466l" (UID: "f4f6cbf1-c75d-4c44-b103-0cac8cb1480e") : secret "dns-default-metrics-tls" not found Jul 31 22:00:11.496213 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.496173 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ae70c5_c782_4984_99b6_77536ebc3f80.slice/crio-5455f6c9491f26f129a4d9f49b8e90cc2c87326d079295e586ecdde162dd635b WatchSource:0}: Error finding container 5455f6c9491f26f129a4d9f49b8e90cc2c87326d079295e586ecdde162dd635b: Status 404 returned error can't find the container with id 5455f6c9491f26f129a4d9f49b8e90cc2c87326d079295e586ecdde162dd635b Jul 31 22:00:11.497014 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.496980 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3d2cae_4201_4a92_abab_c15ff7865fbf.slice/crio-dd0d5451524ab9e364ed33703b3849226a5d502bc48e324532af03de1438c76b WatchSource:0}: Error finding container dd0d5451524ab9e364ed33703b3849226a5d502bc48e324532af03de1438c76b: Status 404 returned error can't find the container with id dd0d5451524ab9e364ed33703b3849226a5d502bc48e324532af03de1438c76b Jul 31 22:00:11.498012 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.497973 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502ad9e6_5e67_417b_9efb_6e1625647fd1.slice/crio-bd531f21b9bc5f51a54b904768b1cd803534adee87ca52ebc7c4677a790e6eea WatchSource:0}: Error finding container bd531f21b9bc5f51a54b904768b1cd803534adee87ca52ebc7c4677a790e6eea: Status 404 returned error can't find the container with id bd531f21b9bc5f51a54b904768b1cd803534adee87ca52ebc7c4677a790e6eea Jul 31 22:00:11.499095 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.499078 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730d0f93_98f3_42a8_9ad2_a153a2cd71f9.slice/crio-2a114e08450e386f492079ba3a77ade3f88717705073fddc27c9fed7e8c89d0b WatchSource:0}: Error finding container 2a114e08450e386f492079ba3a77ade3f88717705073fddc27c9fed7e8c89d0b: Status 404 returned error can't find the container with id 2a114e08450e386f492079ba3a77ade3f88717705073fddc27c9fed7e8c89d0b Jul 31 22:00:11.671396 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.671341 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6cj9w"] Jul 31 22:00:11.674318 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.674264 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22807a73_4c34_4adc_9993_5b3019dbd876.slice/crio-4e2c904e4a86d1d39e8fde21a9b6b1528dc3f4b05fbb5c439b0ed70e87cecbd1 WatchSource:0}: Error finding container 4e2c904e4a86d1d39e8fde21a9b6b1528dc3f4b05fbb5c439b0ed70e87cecbd1: Status 404 returned error can't find the container with id 4e2c904e4a86d1d39e8fde21a9b6b1528dc3f4b05fbb5c439b0ed70e87cecbd1 Jul 31 22:00:11.675612 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.675576 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522"] Jul 31 22:00:11.679125 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.679093 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c31e4e4_71ec_4ce6_9b4f_cc004da9f007.slice/crio-7d6485a75e7015fa3b3d54b2b1c8fb6a3fccc25c64b8231ea2248cf632d4a2c9 WatchSource:0}: Error finding container 7d6485a75e7015fa3b3d54b2b1c8fb6a3fccc25c64b8231ea2248cf632d4a2c9: Status 404 returned error can't find the container with id 7d6485a75e7015fa3b3d54b2b1c8fb6a3fccc25c64b8231ea2248cf632d4a2c9 Jul 31 22:00:11.679937 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.679826 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nw5p"] Jul 31 22:00:11.682757 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.682730 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe077f6_5e11_4e9a_bb32_b6f200b4fd76.slice/crio-5c09461953782ee6b2a786bf423b60c9c5f6b43b25b54377ea9bea2977befee3 WatchSource:0}: Error finding container 5c09461953782ee6b2a786bf423b60c9c5f6b43b25b54377ea9bea2977befee3: Status 404 returned error can't find the container with id 5c09461953782ee6b2a786bf423b60c9c5f6b43b25b54377ea9bea2977befee3 Jul 31 22:00:11.687613 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:11.687540 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m"] Jul 31 22:00:11.869075 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:11.868992 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda19c41c2_668e_495f_aa3f_e9493d1e94c7.slice/crio-1c9af9b8f59ce0e1b0adcbf531697f4a9781ac01c156f1caa53f13a8d6ef4ec6 WatchSource:0}: Error finding container 1c9af9b8f59ce0e1b0adcbf531697f4a9781ac01c156f1caa53f13a8d6ef4ec6: Status 404 returned error can't find the container with id 1c9af9b8f59ce0e1b0adcbf531697f4a9781ac01c156f1caa53f13a8d6ef4ec6 Jul 31 22:00:12.135887 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.135849 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" event={"ID":"a19c41c2-668e-495f-aa3f-e9493d1e94c7","Type":"ContainerStarted","Data":"1c9af9b8f59ce0e1b0adcbf531697f4a9781ac01c156f1caa53f13a8d6ef4ec6"} Jul 31 22:00:12.137064 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.137031 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" event={"ID":"79ae70c5-c782-4984-99b6-77536ebc3f80","Type":"ContainerStarted","Data":"5455f6c9491f26f129a4d9f49b8e90cc2c87326d079295e586ecdde162dd635b"} Jul 31 22:00:12.138147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.138117 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" event={"ID":"502ad9e6-5e67-417b-9efb-6e1625647fd1","Type":"ContainerStarted","Data":"bd531f21b9bc5f51a54b904768b1cd803534adee87ca52ebc7c4677a790e6eea"} Jul 31 22:00:12.139288 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.139267 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nw5p" event={"ID":"efe077f6-5e11-4e9a-bb32-b6f200b4fd76","Type":"ContainerStarted","Data":"5c09461953782ee6b2a786bf423b60c9c5f6b43b25b54377ea9bea2977befee3"} Jul 31 22:00:12.141904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.141880 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerStarted","Data":"e7f9eaac1c007d852572bd442a4f6b282b1c926081dd72cc4e0114c5723c4bf1"} Jul 31 22:00:12.142998 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.142975 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6cj9w" event={"ID":"22807a73-4c34-4adc-9993-5b3019dbd876","Type":"ContainerStarted","Data":"4e2c904e4a86d1d39e8fde21a9b6b1528dc3f4b05fbb5c439b0ed70e87cecbd1"} Jul 31 22:00:12.143961 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.143942 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" event={"ID":"1c31e4e4-71ec-4ce6-9b4f-cc004da9f007","Type":"ContainerStarted","Data":"7d6485a75e7015fa3b3d54b2b1c8fb6a3fccc25c64b8231ea2248cf632d4a2c9"} Jul 31 22:00:12.147386 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.147368 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-m67tr" event={"ID":"5b3d2cae-4201-4a92-abab-c15ff7865fbf","Type":"ContainerStarted","Data":"dd0d5451524ab9e364ed33703b3849226a5d502bc48e324532af03de1438c76b"} Jul 31 22:00:12.148458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:12.148437 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" event={"ID":"730d0f93-98f3-42a8-9ad2-a153a2cd71f9","Type":"ContainerStarted","Data":"2a114e08450e386f492079ba3a77ade3f88717705073fddc27c9fed7e8c89d0b"} Jul 31 22:00:13.061311 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.061054 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:13.061940 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.061517 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 31 22:00:13.061940 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.061535 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbc9d58-vq54h: secret "image-registry-tls" not found Jul 31 22:00:13.061940 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.061607 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls podName:5ce1f101-780d-4c40-b7ce-6d9d7844efc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:17.061574917 +0000 UTC m=+41.626423496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls") pod "image-registry-54bbc9d58-vq54h" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8") : secret "image-registry-tls" not found Jul 31 22:00:13.156782 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.156739 2554 generic.go:356] "Generic (PLEG): container finished" podID="790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0" containerID="e7f9eaac1c007d852572bd442a4f6b282b1c926081dd72cc4e0114c5723c4bf1" exitCode=0 Jul 31 22:00:13.156960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.156791 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerDied","Data":"e7f9eaac1c007d852572bd442a4f6b282b1c926081dd72cc4e0114c5723c4bf1"} Jul 31 22:00:13.162057 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.162027 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:13.162169 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.162125 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:13.162169 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.162162 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:13.162285 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.162200 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:13.162347 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162338 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 22:00:13.162457 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162394 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls podName:5099f668-6faf-4fba-a215-af7fed8091cd nodeName:}" failed. No retries permitted until 2026-07-31 22:00:17.162375323 +0000 UTC m=+41.727223886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-8wzr7" (UID: "5099f668-6faf-4fba-a215-af7fed8091cd") : secret "samples-operator-tls" not found Jul 31 22:00:13.162729 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162694 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 22:00:13.162805 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162737 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:17.162724214 +0000 UTC m=+41.727572777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : secret "router-metrics-certs-default" not found Jul 31 22:00:13.162805 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162791 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:13.162911 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162825 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls podName:22deacc6-c170-4286-9dab-f20d65fe111b nodeName:}" failed. No retries permitted until 2026-07-31 22:00:17.162814283 +0000 UTC m=+41.727662844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-h4d99" (UID: "22deacc6-c170-4286-9dab-f20d65fe111b") : secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:13.162911 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.162886 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:17.162877185 +0000 UTC m=+41.727725748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : configmap references non-existent config key: service-ca.crt Jul 31 22:00:13.466378 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.465688 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:13.466378 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.465829 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 22:00:13.466378 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:13.465884 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls podName:f4f6cbf1-c75d-4c44-b103-0cac8cb1480e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:17.465867295 +0000 UTC m=+42.030715856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls") pod "dns-default-4466l" (UID: "f4f6cbf1-c75d-4c44-b103-0cac8cb1480e") : secret "dns-default-metrics-tls" not found Jul 31 22:00:13.653426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.653374 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:13.750543 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:13.750487 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:14.172762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:14.172704 2554 generic.go:356] "Generic (PLEG): container finished" podID="790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0" containerID="a99deb68f9c793c862a46b5417aba665ff3be604731e5affefae98780fc1f4b4" exitCode=0 Jul 31 22:00:14.173443 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:14.173201 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerDied","Data":"a99deb68f9c793c862a46b5417aba665ff3be604731e5affefae98780fc1f4b4"} Jul 31 22:00:14.179534 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:14.179509 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" event={"ID":"1c31e4e4-71ec-4ce6-9b4f-cc004da9f007","Type":"ContainerStarted","Data":"42e4b4d9def0f187f65a3b2472148daa464a5066b3e7d1f7b8269a87a691745f"} Jul 31 22:00:14.330978 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:14.330889 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:14.566236 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:14.566087 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:14.665743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:14.665694 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:15.187413 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:15.187341 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" event={"ID":"790d18cd-1f65-43c4-b4f7-f7e1dc3ba3a0","Type":"ContainerStarted","Data":"02558338ac881f1fb4ab84401d95ef2c2d40299f3d4d63e41e5e4e43acb11152"} Jul 31 22:00:15.217070 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:15.217010 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-dk522" podStartSLOduration=17.992770423 podStartE2EDuration="20.21698749s" totalImagesPullingTime="2.224217067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:14.217831782 +0000 UTC m=+38.782680365" watchObservedRunningTime="2026-07-31 22:00:15.21698749 +0000 UTC m=+39.781836073" Jul 31 22:00:15.348177 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:15.348088 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:15.450840 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:15.450802 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:17.101160 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:17.101122 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:17.101568 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.101239 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 31 22:00:17.101568 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.101251 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbc9d58-vq54h: secret "image-registry-tls" not found Jul 31 22:00:17.101568 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.101315 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls podName:5ce1f101-780d-4c40-b7ce-6d9d7844efc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.101300898 +0000 UTC m=+49.666149462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls") pod "image-registry-54bbc9d58-vq54h" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8") : secret "image-registry-tls" not found Jul 31 22:00:17.201994 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:17.201964 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:17.202159 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:17.202032 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:17.202159 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202120 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 22:00:17.202159 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202124 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 22:00:17.202159 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:17.202114 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:17.202297 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:17.202166 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:17.202297 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202180 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls podName:5099f668-6faf-4fba-a215-af7fed8091cd nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.202163045 +0000 UTC m=+49.767011605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-8wzr7" (UID: "5099f668-6faf-4fba-a215-af7fed8091cd") : secret "samples-operator-tls" not found Jul 31 22:00:17.202297 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202220 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:17.202297 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202236 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.202199824 +0000 UTC m=+49.767048387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : secret "router-metrics-certs-default" not found Jul 31 22:00:17.202297 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202269 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls podName:22deacc6-c170-4286-9dab-f20d65fe111b nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.202258637 +0000 UTC m=+49.767107198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-h4d99" (UID: "22deacc6-c170-4286-9dab-f20d65fe111b") : secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:17.202297 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.202284 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.202276047 +0000 UTC m=+49.767124606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : configmap references non-existent config key: service-ca.crt Jul 31 22:00:17.505444 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:17.505403 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:17.505638 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.505581 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 22:00:17.505713 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:17.505700 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls podName:f4f6cbf1-c75d-4c44-b103-0cac8cb1480e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.505680591 +0000 UTC m=+50.070529170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls") pod "dns-default-4466l" (UID: "f4f6cbf1-c75d-4c44-b103-0cac8cb1480e") : secret "dns-default-metrics-tls" not found Jul 31 22:00:20.203758 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.203321 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" event={"ID":"a19c41c2-668e-495f-aa3f-e9493d1e94c7","Type":"ContainerStarted","Data":"442190d7793af83a18dcf3aac1d93f17c383876db4fc6a326829a52b98f7f9a2"} Jul 31 22:00:20.206180 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.205898 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" event={"ID":"79ae70c5-c782-4984-99b6-77536ebc3f80","Type":"ContainerStarted","Data":"1e6a1af393a31a1ca05bbec6ed0b8ab4d509c632a65f5ab5b577740f05e394ac"} Jul 31 22:00:20.208546 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.208521 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nw5p" event={"ID":"efe077f6-5e11-4e9a-bb32-b6f200b4fd76","Type":"ContainerStarted","Data":"73b0870deecf6a36bf411a10fe206586e2d4a68f417b099f751b8b56c677ca51"} Jul 31 22:00:20.209265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.209237 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:20.215661 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.215621 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-m67tr" event={"ID":"5b3d2cae-4201-4a92-abab-c15ff7865fbf","Type":"ContainerStarted","Data":"2a07a9f7b5f405346f4477af028a1c39ed8c54bbc18ab6062d11d8f36eb7d592"} Jul 31 22:00:20.220582 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.220496 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" event={"ID":"730d0f93-98f3-42a8-9ad2-a153a2cd71f9","Type":"ContainerStarted","Data":"d0399dc428da9b92618b458c9ef0fc2293025b62f636369420e45cd88c1fb2e5"} Jul 31 22:00:20.229623 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.228679 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zwxm5" podStartSLOduration=16.44788856 podStartE2EDuration="44.228656257s" totalImagesPullingTime="27.780767697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:15.217200207 +0000 UTC m=+39.782048790" watchObservedRunningTime="2026-07-31 22:00:20.228656257 +0000 UTC m=+44.793504839" Jul 31 22:00:20.256880 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.256829 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-m67tr" podStartSLOduration=16.778351052 podStartE2EDuration="25.256796813s" totalImagesPullingTime="8.478445761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:20.25646125 +0000 UTC m=+44.821309837" watchObservedRunningTime="2026-07-31 22:00:20.256796813 +0000 UTC m=+44.821645376" Jul 31 22:00:20.257384 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:20.257353 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8nw5p" podStartSLOduration=35.962716868 podStartE2EDuration="44.257341553s" totalImagesPullingTime="8.294624685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:20.229359358 +0000 UTC m=+44.794207943" watchObservedRunningTime="2026-07-31 22:00:20.257341553 +0000 UTC m=+44.822190135" Jul 31 22:00:21.021929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.021704 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7tmn9"] Jul 31 22:00:21.034355 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.033718 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.035678 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.035621 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7tmn9"] Jul 31 22:00:21.036286 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.036097 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 31 22:00:21.036642 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.036422 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 31 22:00:21.036642 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.036479 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-snlzz\"" Jul 31 22:00:21.039570 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.039532 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:21.108137 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.108103 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:21.143285 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.143257 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h7gg\" (UniqueName: \"kubernetes.io/projected/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-api-access-8h7gg\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.143410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.143353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-crio-socket\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.143462 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.143429 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-data-volume\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.143502 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.143481 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.143593 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.143574 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.226955 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.226909 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6cj9w" event={"ID":"22807a73-4c34-4adc-9993-5b3019dbd876","Type":"ContainerStarted","Data":"454eda5c00b95f487c0f7c92d300ef7d925d4a72847eb75bd38358bd823bc978"} Jul 31 22:00:21.229144 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.229121 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/0.log" Jul 31 22:00:21.229275 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.229165 2554 generic.go:356] "Generic (PLEG): container finished" podID="730d0f93-98f3-42a8-9ad2-a153a2cd71f9" containerID="d0399dc428da9b92618b458c9ef0fc2293025b62f636369420e45cd88c1fb2e5" exitCode=255 Jul 31 22:00:21.229275 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.229193 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" event={"ID":"730d0f93-98f3-42a8-9ad2-a153a2cd71f9","Type":"ContainerDied","Data":"d0399dc428da9b92618b458c9ef0fc2293025b62f636369420e45cd88c1fb2e5"} Jul 31 22:00:21.229452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.229436 2554 scope.go:117] "RemoveContainer" containerID="d0399dc428da9b92618b458c9ef0fc2293025b62f636369420e45cd88c1fb2e5" Jul 31 22:00:21.231770 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.231374 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" event={"ID":"502ad9e6-5e67-417b-9efb-6e1625647fd1","Type":"ContainerStarted","Data":"c5ee9bf659fcdb33cbbd46dd366d9a8100870f2325e136904af4ae7db8ed3c55"} Jul 31 22:00:21.244215 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244191 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h7gg\" (UniqueName: \"kubernetes.io/projected/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-api-access-8h7gg\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244313 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244246 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-crio-socket\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244313 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244295 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-data-volume\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244419 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244332 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244465 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.244438 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 22:00:21.244509 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.244502 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:21.744485407 +0000 UTC m=+46.309333979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : secret "insights-runtime-extractor-tls" not found Jul 31 22:00:21.244564 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244538 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-crio-socket\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244681 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244651 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244752 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.244738 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 22:00:21.244811 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.244784 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-data-volume\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.244863 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.244825 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:21.744806782 +0000 UTC m=+46.309655346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : configmap "kube-rbac-proxy" not found Jul 31 22:00:21.245073 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.245028 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6cj9w" podStartSLOduration=3.941022702 podStartE2EDuration="12.245010974s" totalImagesPullingTime="8.303988272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:21.243515292 +0000 UTC m=+45.808363878" watchObservedRunningTime="2026-07-31 22:00:21.245010974 +0000 UTC m=+45.809859537" Jul 31 22:00:21.255343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.255270 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h7gg\" (UniqueName: \"kubernetes.io/projected/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-api-access-8h7gg\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.279351 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.279305 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" podStartSLOduration=17.793921905 podStartE2EDuration="26.279286647s" totalImagesPullingTime="8.485364742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:21.278725667 +0000 UTC m=+45.843574248" watchObservedRunningTime="2026-07-31 22:00:21.279286647 +0000 UTC m=+45.844135231" Jul 31 22:00:21.291228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.291130 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:21.326898 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.326732 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-br2c4" podStartSLOduration=17.847799244 podStartE2EDuration="26.326715961s" totalImagesPullingTime="8.478916717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:21.32530646 +0000 UTC m=+45.890155043" watchObservedRunningTime="2026-07-31 22:00:21.326715961 +0000 UTC m=+45.891564855" Jul 31 22:00:21.327146 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.327115 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" podStartSLOduration=18.24305741 podStartE2EDuration="26.327106246s" totalImagesPullingTime="8.084048836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:21.305928002 +0000 UTC m=+45.870776584" watchObservedRunningTime="2026-07-31 22:00:21.327106246 +0000 UTC m=+45.891954827" Jul 31 22:00:21.392684 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.392650 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:21.591066 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.590932 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:21.750557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.750521 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.750739 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:21.750578 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:21.750739 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.750665 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 22:00:21.750739 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.750679 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 22:00:21.750739 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.750727 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:22.750710844 +0000 UTC m=+47.315559406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : configmap "kube-rbac-proxy" not found Jul 31 22:00:21.750739 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:21.750740 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:22.750734469 +0000 UTC m=+47.315583029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : secret "insights-runtime-extractor-tls" not found Jul 31 22:00:22.068047 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.067983 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:22.086133 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.086104 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r"] Jul 31 22:00:22.090555 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.090535 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" Jul 31 22:00:22.093024 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.092995 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 31 22:00:22.093156 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.093138 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nwwkz\"" Jul 31 22:00:22.093849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.093834 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 31 22:00:22.100557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.100536 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r"] Jul 31 22:00:22.153105 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.153073 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:22.155164 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.155141 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkqfb\" (UniqueName: \"kubernetes.io/projected/0e25a82a-1ba8-40c3-848d-235082a47ac6-kube-api-access-tkqfb\") pod \"migrator-68bc745d87-sms7r\" (UID: \"0e25a82a-1ba8-40c3-848d-235082a47ac6\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" Jul 31 22:00:22.236730 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.236703 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:00:22.237233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.237110 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/0.log" Jul 31 22:00:22.237233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.237151 2554 generic.go:356] "Generic (PLEG): container finished" podID="730d0f93-98f3-42a8-9ad2-a153a2cd71f9" containerID="86ba2faab92a5372c0e404ea38daea9698e4b167727e1b2d6e86fef32ec0f8e9" exitCode=255 Jul 31 22:00:22.237325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.237260 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" event={"ID":"730d0f93-98f3-42a8-9ad2-a153a2cd71f9","Type":"ContainerDied","Data":"86ba2faab92a5372c0e404ea38daea9698e4b167727e1b2d6e86fef32ec0f8e9"} Jul 31 22:00:22.237325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.237313 2554 scope.go:117] "RemoveContainer" containerID="d0399dc428da9b92618b458c9ef0fc2293025b62f636369420e45cd88c1fb2e5" Jul 31 22:00:22.237459 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.237435 2554 scope.go:117] "RemoveContainer" containerID="86ba2faab92a5372c0e404ea38daea9698e4b167727e1b2d6e86fef32ec0f8e9" Jul 31 22:00:22.237748 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:22.237723 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-bf69l_openshift-console-operator(730d0f93-98f3-42a8-9ad2-a153a2cd71f9)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" podUID="730d0f93-98f3-42a8-9ad2-a153a2cd71f9" Jul 31 22:00:22.252230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.252212 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:22.255845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.255732 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkqfb\" (UniqueName: \"kubernetes.io/projected/0e25a82a-1ba8-40c3-848d-235082a47ac6-kube-api-access-tkqfb\") pod \"migrator-68bc745d87-sms7r\" (UID: \"0e25a82a-1ba8-40c3-848d-235082a47ac6\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" Jul 31 22:00:22.265084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.265065 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkqfb\" (UniqueName: \"kubernetes.io/projected/0e25a82a-1ba8-40c3-848d-235082a47ac6-kube-api-access-tkqfb\") pod \"migrator-68bc745d87-sms7r\" (UID: \"0e25a82a-1ba8-40c3-848d-235082a47ac6\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" Jul 31 22:00:22.384910 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.384837 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xhptq_2ed8dd22-e69c-44e5-8184-1694658e7c85/dns-node-resolver/0.log" Jul 31 22:00:22.399993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.399963 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" Jul 31 22:00:22.551675 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.551648 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r"] Jul 31 22:00:22.554503 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:22.554476 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e25a82a_1ba8_40c3_848d_235082a47ac6.slice/crio-0e7f6f1a8201e8fc2e2ec84af6d9f4b45034f18ca209144a9a53ed7ca36e4b7f WatchSource:0}: Error finding container 0e7f6f1a8201e8fc2e2ec84af6d9f4b45034f18ca209144a9a53ed7ca36e4b7f: Status 404 returned error can't find the container with id 0e7f6f1a8201e8fc2e2ec84af6d9f4b45034f18ca209144a9a53ed7ca36e4b7f Jul 31 22:00:22.762184 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.762145 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:22.762366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:22.762204 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:22.762366 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:22.762282 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 22:00:22.762366 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:22.762301 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 22:00:22.762366 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:22.762335 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:24.762321996 +0000 UTC m=+49.327170556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : configmap "kube-rbac-proxy" not found Jul 31 22:00:22.762366 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:22.762354 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:24.762340244 +0000 UTC m=+49.327188803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : secret "insights-runtime-extractor-tls" not found Jul 31 22:00:23.241987 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.241955 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:00:23.242378 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.242325 2554 scope.go:117] "RemoveContainer" containerID="86ba2faab92a5372c0e404ea38daea9698e4b167727e1b2d6e86fef32ec0f8e9" Jul 31 22:00:23.242575 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:23.242551 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-bf69l_openshift-console-operator(730d0f93-98f3-42a8-9ad2-a153a2cd71f9)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" podUID="730d0f93-98f3-42a8-9ad2-a153a2cd71f9" Jul 31 22:00:23.243211 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.243188 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" event={"ID":"0e25a82a-1ba8-40c3-848d-235082a47ac6","Type":"ContainerStarted","Data":"0e7f6f1a8201e8fc2e2ec84af6d9f4b45034f18ca209144a9a53ed7ca36e4b7f"} Jul 31 22:00:23.384484 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.384457 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vt8r7_b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7/node-ca/0.log" Jul 31 22:00:23.649911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.649830 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-plmtl"] Jul 31 22:00:23.672960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.672927 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-plmtl"] Jul 31 22:00:23.673121 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.673043 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.676564 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.676539 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 31 22:00:23.676705 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.676636 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 31 22:00:23.676775 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.676550 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 31 22:00:23.676844 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.676822 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 31 22:00:23.677002 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.676957 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f2gjk\"" Jul 31 22:00:23.768986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.768959 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:23.771078 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.771054 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4drkc\" (UniqueName: \"kubernetes.io/projected/f206f2e8-55f7-4655-8cae-10b288cff4ea-kube-api-access-4drkc\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.771229 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.771171 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f206f2e8-55f7-4655-8cae-10b288cff4ea-signing-key\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.771295 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.771229 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f206f2e8-55f7-4655-8cae-10b288cff4ea-signing-cabundle\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.786804 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.786777 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6cj9w_22807a73-4c34-4adc-9993-5b3019dbd876/serve-healthcheck-canary/0.log" Jul 31 22:00:23.871561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.871527 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:23.871743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.871567 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f206f2e8-55f7-4655-8cae-10b288cff4ea-signing-key\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.871743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.871626 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f206f2e8-55f7-4655-8cae-10b288cff4ea-signing-cabundle\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.871743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.871696 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4drkc\" (UniqueName: \"kubernetes.io/projected/f206f2e8-55f7-4655-8cae-10b288cff4ea-kube-api-access-4drkc\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.872372 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.872350 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f206f2e8-55f7-4655-8cae-10b288cff4ea-signing-cabundle\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.874586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.874566 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f206f2e8-55f7-4655-8cae-10b288cff4ea-signing-key\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.882749 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.882731 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4drkc\" (UniqueName: \"kubernetes.io/projected/f206f2e8-55f7-4655-8cae-10b288cff4ea-kube-api-access-4drkc\") pod \"service-ca-98c88fbb5-plmtl\" (UID: \"f206f2e8-55f7-4655-8cae-10b288cff4ea\") " pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:23.984807 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:23.984781 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" Jul 31 22:00:24.049458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.049426 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz"] Jul 31 22:00:24.064220 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.064197 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.065501 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.065479 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz"] Jul 31 22:00:24.066893 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.066878 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2t625\"" Jul 31 22:00:24.067406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.067386 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 31 22:00:24.067513 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.067453 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 31 22:00:24.175060 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.175032 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf66033c-d967-4619-a9d3-bf7b399653b1-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.175184 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.175072 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.225502 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.225471 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-plmtl"] Jul 31 22:00:24.227952 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:24.227925 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf206f2e8_55f7_4655_8cae_10b288cff4ea.slice/crio-6a3432b739c4fe45820467dacf2f925c381c005cd0ead64a8b97877c9b19cea2 WatchSource:0}: Error finding container 6a3432b739c4fe45820467dacf2f925c381c005cd0ead64a8b97877c9b19cea2: Status 404 returned error can't find the container with id 6a3432b739c4fe45820467dacf2f925c381c005cd0ead64a8b97877c9b19cea2 Jul 31 22:00:24.247579 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.247548 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" event={"ID":"0e25a82a-1ba8-40c3-848d-235082a47ac6","Type":"ContainerStarted","Data":"699c49ba8013937e232d47fef8d42034aeb83c2b73f0c7dee60733b02ecd2bb1"} Jul 31 22:00:24.248636 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.248613 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" event={"ID":"f206f2e8-55f7-4655-8cae-10b288cff4ea","Type":"ContainerStarted","Data":"6a3432b739c4fe45820467dacf2f925c381c005cd0ead64a8b97877c9b19cea2"} Jul 31 22:00:24.275906 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.275881 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf66033c-d967-4619-a9d3-bf7b399653b1-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.276000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.275910 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.276038 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.276020 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 22:00:24.276080 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.276064 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert podName:cf66033c-d967-4619-a9d3-bf7b399653b1 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:24.776050933 +0000 UTC m=+49.340899493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-4bdbz" (UID: "cf66033c-d967-4619-a9d3-bf7b399653b1") : secret "networking-console-plugin-cert" not found Jul 31 22:00:24.276424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.276405 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf66033c-d967-4619-a9d3-bf7b399653b1-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.553900 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.553804 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:24.648020 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.647987 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:24.780090 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.780049 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:24.780234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.780126 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:24.780234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.780155 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:24.780234 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.780215 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 22:00:24.780350 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.780255 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 22:00:24.780350 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.780278 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:28.780260097 +0000 UTC m=+53.345108669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : secret "insights-runtime-extractor-tls" not found Jul 31 22:00:24.780350 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.780310 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:28.780297094 +0000 UTC m=+53.345145653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : configmap "kube-rbac-proxy" not found Jul 31 22:00:24.780350 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.780309 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 22:00:24.780473 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:24.780353 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert podName:cf66033c-d967-4619-a9d3-bf7b399653b1 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:25.780342372 +0000 UTC m=+50.345190945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-4bdbz" (UID: "cf66033c-d967-4619-a9d3-bf7b399653b1") : secret "networking-console-plugin-cert" not found Jul 31 22:00:24.910531 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:24.910448 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:25.185136 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.185105 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:25.185292 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.185274 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 31 22:00:25.185329 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.185300 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54bbc9d58-vq54h: secret "image-registry-tls" not found Jul 31 22:00:25.185360 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.185352 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls podName:5ce1f101-780d-4c40-b7ce-6d9d7844efc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.185337734 +0000 UTC m=+65.750186294 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls") pod "image-registry-54bbc9d58-vq54h" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8") : secret "image-registry-tls" not found Jul 31 22:00:25.253410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.253371 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" event={"ID":"0e25a82a-1ba8-40c3-848d-235082a47ac6","Type":"ContainerStarted","Data":"c4c524eb244fa5f4abd69ae711059e15acbd0bd5e8cdee800f2f13738ced10f2"} Jul 31 22:00:25.254924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.254900 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" event={"ID":"f206f2e8-55f7-4655-8cae-10b288cff4ea","Type":"ContainerStarted","Data":"e346cb1129fedf1b0d20b70b58b19499adb792378322a6f081c530a5a7aeed4f"} Jul 31 22:00:25.285828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.285800 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:25.285966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.285833 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:25.285966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.285874 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:25.285966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.285919 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.285981 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.285964265 +0000 UTC m=+65.850812829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : configmap references non-existent config key: service-ca.crt Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.286024 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.286031 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.286034 2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.286068 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs podName:07c72711-0eaf-4064-b343-3572c0783bf0 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.286053993 +0000 UTC m=+65.850902573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs") pod "router-default-644f8b4f6-vsfsr" (UID: "07c72711-0eaf-4064-b343-3572c0783bf0") : secret "router-metrics-certs-default" not found Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.286092 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls podName:5099f668-6faf-4fba-a215-af7fed8091cd nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.286076464 +0000 UTC m=+65.850925032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-8wzr7" (UID: "5099f668-6faf-4fba-a215-af7fed8091cd") : secret "samples-operator-tls" not found Jul 31 22:00:25.286124 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.286110 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls podName:22deacc6-c170-4286-9dab-f20d65fe111b nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.286100428 +0000 UTC m=+65.850948995 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-h4d99" (UID: "22deacc6-c170-4286-9dab-f20d65fe111b") : secret "cluster-monitoring-operator-tls" not found Jul 31 22:00:25.300171 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.300135 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-sms7r" podStartSLOduration=1.755141512 podStartE2EDuration="3.300121121s" totalImagesPullingTime="1.544979609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:25.277121959 +0000 UTC m=+49.841970541" watchObservedRunningTime="2026-07-31 22:00:25.300121121 +0000 UTC m=+49.864969706" Jul 31 22:00:25.300408 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.300390 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-plmtl" podStartSLOduration=2.300386274 podStartE2EDuration="2.300386274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:25.299701319 +0000 UTC m=+49.864549901" watchObservedRunningTime="2026-07-31 22:00:25.300386274 +0000 UTC m=+49.865234867" Jul 31 22:00:25.589346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.589258 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:25.589500 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.589418 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 31 22:00:25.589500 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.589483 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls podName:f4f6cbf1-c75d-4c44-b103-0cac8cb1480e nodeName:}" failed. No retries permitted until 2026-07-31 22:00:41.58946792 +0000 UTC m=+66.154316480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls") pod "dns-default-4466l" (UID: "f4f6cbf1-c75d-4c44-b103-0cac8cb1480e") : secret "dns-default-metrics-tls" not found Jul 31 22:00:25.730167 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.730131 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:25.790772 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.790739 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:25.790962 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.790910 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 22:00:25.791043 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:25.791001 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert podName:cf66033c-d967-4619-a9d3-bf7b399653b1 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:27.790977911 +0000 UTC m=+52.355826474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-4bdbz" (UID: "cf66033c-d967-4619-a9d3-bf7b399653b1") : secret "networking-console-plugin-cert" not found Jul 31 22:00:25.936650 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:25.936620 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:26.032276 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:26.032133 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:26.149467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:26.149425 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:26.368224 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:26.368138 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:26.456498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:26.456463 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:27.173332 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:27.173287 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:27.791982 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:27.791944 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:27.809977 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:27.809949 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:27.810109 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:27.810083 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 22:00:27.810166 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:27.810148 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert podName:cf66033c-d967-4619-a9d3-bf7b399653b1 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:31.810126469 +0000 UTC m=+56.374975054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-4bdbz" (UID: "cf66033c-d967-4619-a9d3-bf7b399653b1") : secret "networking-console-plugin-cert" not found Jul 31 22:00:27.987727 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:27.987688 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:28.395076 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:28.395048 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:28.587262 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:28.587222 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:28.819115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:28.819081 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:28.819514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:28.819143 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:28.819514 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:28.819245 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 31 22:00:28.819514 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:28.819256 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 31 22:00:28.819514 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:28.819307 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:36.819294359 +0000 UTC m=+61.384142919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : configmap "kube-rbac-proxy" not found Jul 31 22:00:28.819514 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:28.819319 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls podName:d3f3f02b-f0e4-4ef6-8ab7-3663abde1396 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:36.81931427 +0000 UTC m=+61.384162830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7tmn9" (UID: "d3f3f02b-f0e4-4ef6-8ab7-3663abde1396") : secret "insights-runtime-extractor-tls" not found Jul 31 22:00:29.082111 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.082017 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:29.180037 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.179991 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:29.376984 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.376877 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:29.449800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.449756 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:29.698806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.698760 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:29.698982 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.698868 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:29.699224 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.699213 2554 scope.go:117] "RemoveContainer" containerID="86ba2faab92a5372c0e404ea38daea9698e4b167727e1b2d6e86fef32ec0f8e9" Jul 31 22:00:29.699371 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:29.699353 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-bf69l_openshift-console-operator(730d0f93-98f3-42a8-9ad2-a153a2cd71f9)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" podUID="730d0f93-98f3-42a8-9ad2-a153a2cd71f9" Jul 31 22:00:29.786683 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.786636 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:29.884496 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:29.884452 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:30.928226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:30.928174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.027229 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.027194 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.176095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.176060 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.276715 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.276679 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.378495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.378460 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.486369 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.486330 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.583009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.582916 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.843360 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.843270 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:31.849053 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.849030 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:31.849190 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:31.849168 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 31 22:00:31.849307 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:31.849232 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert podName:cf66033c-d967-4619-a9d3-bf7b399653b1 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:39.849217749 +0000 UTC m=+64.414066308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-4bdbz" (UID: "cf66033c-d967-4619-a9d3-bf7b399653b1") : secret "networking-console-plugin-cert" not found Jul 31 22:00:31.965421 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:31.965378 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:32.589146 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:32.589110 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:32.787037 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:32.786998 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:32.980158 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:32.980120 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:33.108154 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:33.108106 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:33.382056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:33.381969 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:33.990170 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:33.990136 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:34.203203 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:34.203174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:34.436355 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:34.436319 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:34.745086 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:34.744998 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:34.997119 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:34.997045 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:35.094736 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:35.094702 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:35.377703 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:35.377623 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:35.870905 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:35.870845 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:35.999379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:35.999308 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:36.129348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.129277 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fmtfd" Jul 31 22:00:36.181193 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.181157 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:36.387616 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.387572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:36.897192 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.897151 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:36.897348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.897328 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:36.897794 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.897771 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:36.899544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.899522 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3f3f02b-f0e4-4ef6-8ab7-3663abde1396-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tmn9\" (UID: \"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396\") " pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:36.948829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.948796 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-snlzz\"" Jul 31 22:00:36.957088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:36.957059 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7tmn9" Jul 31 22:00:37.086296 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:37.086261 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7tmn9"] Jul 31 22:00:37.089866 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:37.089837 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f3f02b_f0e4_4ef6_8ab7_3663abde1396.slice/crio-1807559e9c75d3db51e7d2785936d758b6826024654ed1f42f28ab0a4864bbd9 WatchSource:0}: Error finding container 1807559e9c75d3db51e7d2785936d758b6826024654ed1f42f28ab0a4864bbd9: Status 404 returned error can't find the container with id 1807559e9c75d3db51e7d2785936d758b6826024654ed1f42f28ab0a4864bbd9 Jul 31 22:00:37.297202 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:37.297172 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tmn9" event={"ID":"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396","Type":"ContainerStarted","Data":"04abfd92f61082e0901834dd4a86a549a61ab444a534412a18b1a8564a560268"} Jul 31 22:00:37.297359 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:37.297224 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tmn9" event={"ID":"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396","Type":"ContainerStarted","Data":"1807559e9c75d3db51e7d2785936d758b6826024654ed1f42f28ab0a4864bbd9"} Jul 31 22:00:38.302295 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:38.302261 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tmn9" event={"ID":"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396","Type":"ContainerStarted","Data":"148cdd962bbe27f0204961a79f38d3d8a7e2692df0b458e0a7dd934c7c63ef92"} Jul 31 22:00:39.927633 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:39.927587 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:39.929803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:39.929781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf66033c-d967-4619-a9d3-bf7b399653b1-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-4bdbz\" (UID: \"cf66033c-d967-4619-a9d3-bf7b399653b1\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:39.977047 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:39.977029 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2t625\"" Jul 31 22:00:39.985274 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:39.985252 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" Jul 31 22:00:40.113269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:40.113244 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz"] Jul 31 22:00:40.115501 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:40.115474 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf66033c_d967_4619_a9d3_bf7b399653b1.slice/crio-516e9edbc4721d28c62467fb25ffe1b3bb4ace045999a4a1ba4571638196640d WatchSource:0}: Error finding container 516e9edbc4721d28c62467fb25ffe1b3bb4ace045999a4a1ba4571638196640d: Status 404 returned error can't find the container with id 516e9edbc4721d28c62467fb25ffe1b3bb4ace045999a4a1ba4571638196640d Jul 31 22:00:40.310290 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:40.310253 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tmn9" event={"ID":"d3f3f02b-f0e4-4ef6-8ab7-3663abde1396","Type":"ContainerStarted","Data":"a15b49a2f235fd0f4a16d924af8fda89cfebb92d441a6c7ed1a5d47127d7eb13"} Jul 31 22:00:40.311390 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:40.311365 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" event={"ID":"cf66033c-d967-4619-a9d3-bf7b399653b1","Type":"ContainerStarted","Data":"516e9edbc4721d28c62467fb25ffe1b3bb4ace045999a4a1ba4571638196640d"} Jul 31 22:00:40.332067 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:40.332022 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7tmn9" podStartSLOduration=17.59115316 podStartE2EDuration="20.332008108s" totalImagesPullingTime="2.740854948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:00:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:40.330868547 +0000 UTC m=+64.895717129" watchObservedRunningTime="2026-07-31 22:00:40.332008108 +0000 UTC m=+64.896856726" Jul 31 22:00:41.239400 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.239357 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:41.241569 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.241546 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"image-registry-54bbc9d58-vq54h\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:41.316491 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.316460 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" event={"ID":"cf66033c-d967-4619-a9d3-bf7b399653b1","Type":"ContainerStarted","Data":"2d4cacf1ae3426516eeba36c7767601824730c2cb62f66c9bdde6d8b46940e5c"} Jul 31 22:00:41.332050 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.332005 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-4bdbz" podStartSLOduration=16.219302055 podStartE2EDuration="17.331991608s" totalImagesPullingTime="1.112689553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:41.331628241 +0000 UTC m=+65.896476820" watchObservedRunningTime="2026-07-31 22:00:41.331991608 +0000 UTC m=+65.896840190" Jul 31 22:00:41.339976 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.339948 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:41.340103 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.340009 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:41.340103 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.340041 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:41.340103 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.340082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:41.340628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.340591 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c72711-0eaf-4064-b343-3572c0783bf0-service-ca-bundle\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:41.342267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.342240 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07c72711-0eaf-4064-b343-3572c0783bf0-metrics-certs\") pod \"router-default-644f8b4f6-vsfsr\" (UID: \"07c72711-0eaf-4064-b343-3572c0783bf0\") " pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:41.342369 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.342294 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22deacc6-c170-4286-9dab-f20d65fe111b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-h4d99\" (UID: \"22deacc6-c170-4286-9dab-f20d65fe111b\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:41.342369 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.342332 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5099f668-6faf-4fba-a215-af7fed8091cd-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-8wzr7\" (UID: \"5099f668-6faf-4fba-a215-af7fed8091cd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:41.446094 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.446058 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-55g2j\"" Jul 31 22:00:41.454251 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.454216 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:41.458961 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.458945 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k8l2r\"" Jul 31 22:00:41.467200 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.467179 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" Jul 31 22:00:41.524714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.524533 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-v7c68\"" Jul 31 22:00:41.531967 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.531938 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" Jul 31 22:00:41.543119 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.543100 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tjjh6\"" Jul 31 22:00:41.551110 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.551089 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:41.595572 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.595503 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54bbc9d58-vq54h"] Jul 31 22:00:41.600454 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:41.600419 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce1f101_780d_4c40_b7ce_6d9d7844efc8.slice/crio-c149bfcc387c1c187019459129f87b6f95fc25b5a3ad6aadb699db517d45ce38 WatchSource:0}: Error finding container c149bfcc387c1c187019459129f87b6f95fc25b5a3ad6aadb699db517d45ce38: Status 404 returned error can't find the container with id c149bfcc387c1c187019459129f87b6f95fc25b5a3ad6aadb699db517d45ce38 Jul 31 22:00:41.617934 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.617893 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7"] Jul 31 22:00:41.642617 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.642565 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:41.642725 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.642650 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:41.645473 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.645354 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 31 22:00:41.645473 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.645382 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4f6cbf1-c75d-4c44-b103-0cac8cb1480e-metrics-tls\") pod \"dns-default-4466l\" (UID: \"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e\") " pod="openshift-dns/dns-default-4466l" Jul 31 22:00:41.655660 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.655622 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a473fec7-53b1-4440-b7cf-5cdaf810c897-original-pull-secret\") pod \"global-pull-secret-syncer-hkdcf\" (UID: \"a473fec7-53b1-4440-b7cf-5cdaf810c897\") " pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:41.695954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.695928 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99"] Jul 31 22:00:41.697513 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:41.697490 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22deacc6_c170_4286_9dab_f20d65fe111b.slice/crio-361fb3efe9fbd86012b6b7ae7b6a58fe138283f6dbc89a8f767c8efc98ceda15 WatchSource:0}: Error finding container 361fb3efe9fbd86012b6b7ae7b6a58fe138283f6dbc89a8f767c8efc98ceda15: Status 404 returned error can't find the container with id 361fb3efe9fbd86012b6b7ae7b6a58fe138283f6dbc89a8f767c8efc98ceda15 Jul 31 22:00:41.725922 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.725885 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-644f8b4f6-vsfsr"] Jul 31 22:00:41.728817 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:41.728793 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c72711_0eaf_4064_b343_3572c0783bf0.slice/crio-d839383a9c865f33ec835f2a41bc99ebd5f3e0f2906cf94bb355fd2d66be4d45 WatchSource:0}: Error finding container d839383a9c865f33ec835f2a41bc99ebd5f3e0f2906cf94bb355fd2d66be4d45: Status 404 returned error can't find the container with id d839383a9c865f33ec835f2a41bc99ebd5f3e0f2906cf94bb355fd2d66be4d45 Jul 31 22:00:41.743470 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.743448 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:41.746455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.746433 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 31 22:00:41.755877 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.755825 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e-metrics-certs\") pod \"network-metrics-daemon-qqgnd\" (UID: \"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e\") " pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:41.770001 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.769980 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sb6cn\"" Jul 31 22:00:41.773206 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.773182 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hkdcf" Jul 31 22:00:41.777414 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.777394 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqgnd" Jul 31 22:00:41.853870 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.853838 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jdkvx\"" Jul 31 22:00:41.861138 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.861110 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4466l" Jul 31 22:00:41.946504 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.946467 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hkdcf"] Jul 31 22:00:41.951901 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:41.951870 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda473fec7_53b1_4440_b7cf_5cdaf810c897.slice/crio-eaeb5d50d2896a25d4b2220c5e99e23f2d29f6f7d22ca811a288340514a1e065 WatchSource:0}: Error finding container eaeb5d50d2896a25d4b2220c5e99e23f2d29f6f7d22ca811a288340514a1e065: Status 404 returned error can't find the container with id eaeb5d50d2896a25d4b2220c5e99e23f2d29f6f7d22ca811a288340514a1e065 Jul 31 22:00:41.970218 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:41.970134 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qqgnd"] Jul 31 22:00:41.981720 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:41.981675 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4dca04f_d57f_40e9_9adf_ae1e6d56ab0e.slice/crio-d9b09c304ccf2f6d95d28d78bddc502b2af46e6358c101b2c39a76dd891fcaa8 WatchSource:0}: Error finding container d9b09c304ccf2f6d95d28d78bddc502b2af46e6358c101b2c39a76dd891fcaa8: Status 404 returned error can't find the container with id d9b09c304ccf2f6d95d28d78bddc502b2af46e6358c101b2c39a76dd891fcaa8 Jul 31 22:00:42.027797 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.027707 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4466l"] Jul 31 22:00:42.031914 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:42.031888 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f6cbf1_c75d_4c44_b103_0cac8cb1480e.slice/crio-b962dc298a771dd12a8919082a3cc550b50dcde8e3f0bc048bf3f88f3ff93806 WatchSource:0}: Error finding container b962dc298a771dd12a8919082a3cc550b50dcde8e3f0bc048bf3f88f3ff93806: Status 404 returned error can't find the container with id b962dc298a771dd12a8919082a3cc550b50dcde8e3f0bc048bf3f88f3ff93806 Jul 31 22:00:42.321481 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.321398 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" event={"ID":"5ce1f101-780d-4c40-b7ce-6d9d7844efc8","Type":"ContainerStarted","Data":"469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d"} Jul 31 22:00:42.321481 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.321449 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" event={"ID":"5ce1f101-780d-4c40-b7ce-6d9d7844efc8","Type":"ContainerStarted","Data":"c149bfcc387c1c187019459129f87b6f95fc25b5a3ad6aadb699db517d45ce38"} Jul 31 22:00:42.322023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.321498 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:00:42.322695 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.322671 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" event={"ID":"5099f668-6faf-4fba-a215-af7fed8091cd","Type":"ContainerStarted","Data":"6913e0998f2b85ff1f4b5fb4644f041ac513c5993598f26d72a542e686912ac5"} Jul 31 22:00:42.323907 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.323888 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" event={"ID":"07c72711-0eaf-4064-b343-3572c0783bf0","Type":"ContainerStarted","Data":"70f35e2f7e0251ad9b971231dbcf7c4fa61d7d4de647801fa22a34c6091762f4"} Jul 31 22:00:42.323991 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.323917 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" event={"ID":"07c72711-0eaf-4064-b343-3572c0783bf0","Type":"ContainerStarted","Data":"d839383a9c865f33ec835f2a41bc99ebd5f3e0f2906cf94bb355fd2d66be4d45"} Jul 31 22:00:42.324953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.324932 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hkdcf" event={"ID":"a473fec7-53b1-4440-b7cf-5cdaf810c897","Type":"ContainerStarted","Data":"eaeb5d50d2896a25d4b2220c5e99e23f2d29f6f7d22ca811a288340514a1e065"} Jul 31 22:00:42.326033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.326012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" event={"ID":"22deacc6-c170-4286-9dab-f20d65fe111b","Type":"ContainerStarted","Data":"361fb3efe9fbd86012b6b7ae7b6a58fe138283f6dbc89a8f767c8efc98ceda15"} Jul 31 22:00:42.327091 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.327069 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4466l" event={"ID":"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e","Type":"ContainerStarted","Data":"b962dc298a771dd12a8919082a3cc550b50dcde8e3f0bc048bf3f88f3ff93806"} Jul 31 22:00:42.328106 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.328090 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqgnd" event={"ID":"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e","Type":"ContainerStarted","Data":"d9b09c304ccf2f6d95d28d78bddc502b2af46e6358c101b2c39a76dd891fcaa8"} Jul 31 22:00:42.351700 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.351653 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" podStartSLOduration=66.351637984 podStartE2EDuration="1m6.351637984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:42.350530135 +0000 UTC m=+66.915378719" watchObservedRunningTime="2026-07-31 22:00:42.351637984 +0000 UTC m=+66.916486566" Jul 31 22:00:42.371545 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.371499 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" podStartSLOduration=47.371484013 podStartE2EDuration="47.371484013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:42.370693121 +0000 UTC m=+66.935541702" watchObservedRunningTime="2026-07-31 22:00:42.371484013 +0000 UTC m=+66.936332594" Jul 31 22:00:42.552291 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.552255 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:42.555083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.555051 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:42.951846 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:42.951812 2554 scope.go:117] "RemoveContainer" containerID="86ba2faab92a5372c0e404ea38daea9698e4b167727e1b2d6e86fef32ec0f8e9" Jul 31 22:00:43.340676 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:43.339859 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:00:43.340676 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:43.340339 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" event={"ID":"730d0f93-98f3-42a8-9ad2-a153a2cd71f9","Type":"ContainerStarted","Data":"7eb59b013e3d8401280ce2d5c11a183909345a021cf9b85b9b84fb937069cfae"} Jul 31 22:00:43.342374 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:43.341782 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:43.342374 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:43.341893 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:43.343261 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:43.343220 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-644f8b4f6-vsfsr" Jul 31 22:00:43.374068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:43.373131 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" podStartSLOduration=39.89679658 podStartE2EDuration="48.373044948s" totalImagesPullingTime="8.476248368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:43.372860819 +0000 UTC m=+67.937709430" watchObservedRunningTime="2026-07-31 22:00:43.373044948 +0000 UTC m=+67.937893532" Jul 31 22:00:44.261426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:44.260944 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-bf69l" Jul 31 22:00:44.481234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:44.481196 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:44.576478 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:44.576362 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:44.762126 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:44.762094 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:45.170747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:45.170665 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:45.311393 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:45.311360 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:45.816336 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:45.816251 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:45.918128 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:45.918093 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:46.897779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:46.897682 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:47.001471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.001422 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:47.176319 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.176282 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:47.359787 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.359685 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4466l" event={"ID":"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e","Type":"ContainerStarted","Data":"522b472dde59040c3e9a8f04e3ff1a287f1582c35085ac7ebba1c30a5695f165"} Jul 31 22:00:47.359954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.359796 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4466l" Jul 31 22:00:47.359954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.359823 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4466l" event={"ID":"f4f6cbf1-c75d-4c44-b103-0cac8cb1480e","Type":"ContainerStarted","Data":"1d363d1810ce66885f14b1038d92b93b7ff94950ddda0f1509507c8a072c88d1"} Jul 31 22:00:47.361745 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.361718 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqgnd" event={"ID":"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e","Type":"ContainerStarted","Data":"df6799166ef30065c799f666d7a1fd418af95ff82c8a8ebfa2c03850d1022cc9"} Jul 31 22:00:47.361866 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.361771 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqgnd" event={"ID":"d4dca04f-d57f-40e9-9adf-ae1e6d56ab0e","Type":"ContainerStarted","Data":"a91555858c58de80d13fba4c8aa7d969e8087b094b292871cb98a2403f1c245e"} Jul 31 22:00:47.363458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.363435 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" event={"ID":"5099f668-6faf-4fba-a215-af7fed8091cd","Type":"ContainerStarted","Data":"ad319dad750fdfbbdf1ba8e204ea5359f69869e9fefacd145194accc9425e6d5"} Jul 31 22:00:47.363568 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.363484 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" event={"ID":"5099f668-6faf-4fba-a215-af7fed8091cd","Type":"ContainerStarted","Data":"4be23d1b38a09d519e06bee01b50f7e3b5cbcfbf3107fa78f476e1d2b565b9ed"} Jul 31 22:00:47.364948 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.364928 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hkdcf" event={"ID":"a473fec7-53b1-4440-b7cf-5cdaf810c897","Type":"ContainerStarted","Data":"45e39a6926c8db3450ed0a408b35f2e5b03f530f2faabd972e77f00e653bb4f0"} Jul 31 22:00:47.366553 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.366533 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" event={"ID":"22deacc6-c170-4286-9dab-f20d65fe111b","Type":"ContainerStarted","Data":"af0380180ea50e73f992e87b6ddfed1d370fbe9459959fc2e292a1ae7d8a887b"} Jul 31 22:00:47.377329 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.377309 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:47.382645 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.382593 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4466l" podStartSLOduration=33.560801845 podStartE2EDuration="38.382579065s" totalImagesPullingTime="4.82177722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:47.381852898 +0000 UTC m=+71.946701479" watchObservedRunningTime="2026-07-31 22:00:47.382579065 +0000 UTC m=+71.947427646" Jul 31 22:00:47.407471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.407411 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qqgnd" podStartSLOduration=66.538928024 podStartE2EDuration="1m11.40739149s" totalImagesPullingTime="4.868463466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:47.406676076 +0000 UTC m=+71.971524658" watchObservedRunningTime="2026-07-31 22:00:47.40739149 +0000 UTC m=+71.972240073" Jul 31 22:00:47.471694 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.471639 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hkdcf" podStartSLOduration=66.503755831 podStartE2EDuration="1m11.471623663s" totalImagesPullingTime="4.967867832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:47.471232114 +0000 UTC m=+72.036080695" watchObservedRunningTime="2026-07-31 22:00:47.471623663 +0000 UTC m=+72.036472225" Jul 31 22:00:47.510766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.510708 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-8wzr7" podStartSLOduration=47.329750599 podStartE2EDuration="52.510687841s" totalImagesPullingTime="5.180937242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:47.509669033 +0000 UTC m=+72.074517616" watchObservedRunningTime="2026-07-31 22:00:47.510687841 +0000 UTC m=+72.075536425" Jul 31 22:00:47.535092 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.535058 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:47.541976 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.541934 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-h4d99" podStartSLOduration=47.388042117 podStartE2EDuration="52.541921218s" totalImagesPullingTime="5.153879101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 21:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:47.540504241 +0000 UTC m=+72.105352823" watchObservedRunningTime="2026-07-31 22:00:47.541921218 +0000 UTC m=+72.106769817" Jul 31 22:00:47.595122 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.595091 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65"] Jul 31 22:00:47.599090 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.599062 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj"] Jul 31 22:00:47.599246 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.599189 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.602477 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.602446 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.613453 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.613392 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 31 22:00:47.613583 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.613551 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 31 22:00:47.613735 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.613697 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 31 22:00:47.613978 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.613956 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 31 22:00:47.614177 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.614128 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 31 22:00:47.615174 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.615150 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6"] Jul 31 22:00:47.616814 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.616796 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 31 22:00:47.618226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.618167 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.619339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.619321 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 31 22:00:47.619831 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.619811 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 31 22:00:47.627122 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.627101 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 31 22:00:47.627259 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.627240 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-f7z7s\"" Jul 31 22:00:47.646300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.646273 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj"] Jul 31 22:00:47.647497 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.647471 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65"] Jul 31 22:00:47.691174 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.690082 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6"] Jul 31 22:00:47.699781 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.699750 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qlpp\" (UniqueName: \"kubernetes.io/projected/06053773-ff75-413f-8aec-8f7e2a369f88-kube-api-access-5qlpp\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.700002 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.699983 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-ca\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.700130 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.700113 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06053773-ff75-413f-8aec-8f7e2a369f88-tmp\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.700311 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.700291 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.700495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.700478 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/472a3167-50e0-424a-8dc9-2fd90c55fc57-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.700624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.700591 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-hub\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.700784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.700766 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.700927 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.700911 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/06053773-ff75-413f-8aec-8f7e2a369f88-klusterlet-config\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.701088 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.701034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq96n\" (UniqueName: \"kubernetes.io/projected/472a3167-50e0-424a-8dc9-2fd90c55fc57-kube-api-access-dq96n\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.708309 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.708283 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-46znc"] Jul 31 22:00:47.711922 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.711902 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:00:47.718751 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.718725 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 31 22:00:47.718883 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.718808 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 31 22:00:47.718883 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.718725 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n285j\"" Jul 31 22:00:47.747065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.747025 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-46znc"] Jul 31 22:00:47.798426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.798394 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804616 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qlpp\" (UniqueName: \"kubernetes.io/projected/06053773-ff75-413f-8aec-8f7e2a369f88-kube-api-access-5qlpp\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804670 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-ca\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804704 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06053773-ff75-413f-8aec-8f7e2a369f88-tmp\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804736 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804776 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/472a3167-50e0-424a-8dc9-2fd90c55fc57-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804806 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-hub\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804841 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxbg4\" (UniqueName: \"kubernetes.io/projected/91e66e36-c614-401b-b57d-3cfd4257849e-kube-api-access-bxbg4\") pod \"downloads-77c974ffcf-46znc\" (UID: \"91e66e36-c614-401b-b57d-3cfd4257849e\") " pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804891 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804947 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxjt4\" (UniqueName: \"kubernetes.io/projected/8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3-kube-api-access-bxjt4\") pod \"managed-serviceaccount-addon-agent-7554888dcd-rfbg6\" (UID: \"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.804984 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/06053773-ff75-413f-8aec-8f7e2a369f88-klusterlet-config\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.805017 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq96n\" (UniqueName: \"kubernetes.io/projected/472a3167-50e0-424a-8dc9-2fd90c55fc57-kube-api-access-dq96n\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.807402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.805042 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7554888dcd-rfbg6\" (UID: \"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.808137 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.807543 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06053773-ff75-413f-8aec-8f7e2a369f88-tmp\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.808137 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.807894 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/472a3167-50e0-424a-8dc9-2fd90c55fc57-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.808551 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.808487 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.809419 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.809397 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-hub\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.810766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.810737 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/06053773-ff75-413f-8aec-8f7e2a369f88-klusterlet-config\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.810766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.810751 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-ca\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.810922 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.810799 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/472a3167-50e0-424a-8dc9-2fd90c55fc57-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.813249 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.813230 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qlpp\" (UniqueName: \"kubernetes.io/projected/06053773-ff75-413f-8aec-8f7e2a369f88-kube-api-access-5qlpp\") pod \"klusterlet-addon-workmgr-758978cbdd-ghzlj\" (UID: \"06053773-ff75-413f-8aec-8f7e2a369f88\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.820735 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.820715 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq96n\" (UniqueName: \"kubernetes.io/projected/472a3167-50e0-424a-8dc9-2fd90c55fc57-kube-api-access-dq96n\") pod \"cluster-proxy-proxy-agent-5c95cd4f6b-4gp65\" (UID: \"472a3167-50e0-424a-8dc9-2fd90c55fc57\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.869378 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.869291 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq"] Jul 31 22:00:47.873809 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.873785 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:47.876447 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.876419 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 31 22:00:47.876447 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.876435 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vgchh\"" Jul 31 22:00:47.887710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.887687 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq"] Jul 31 22:00:47.905542 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.905514 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxjt4\" (UniqueName: \"kubernetes.io/projected/8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3-kube-api-access-bxjt4\") pod \"managed-serviceaccount-addon-agent-7554888dcd-rfbg6\" (UID: \"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.905979 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.905612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7554888dcd-rfbg6\" (UID: \"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.905979 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.905692 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxbg4\" (UniqueName: \"kubernetes.io/projected/91e66e36-c614-401b-b57d-3cfd4257849e-kube-api-access-bxbg4\") pod \"downloads-77c974ffcf-46znc\" (UID: \"91e66e36-c614-401b-b57d-3cfd4257849e\") " pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:00:47.909390 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.909354 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7554888dcd-rfbg6\" (UID: \"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.920732 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.920706 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" Jul 31 22:00:47.922493 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.922469 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxjt4\" (UniqueName: \"kubernetes.io/projected/8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3-kube-api-access-bxjt4\") pod \"managed-serviceaccount-addon-agent-7554888dcd-rfbg6\" (UID: \"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.928793 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.928768 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:47.935143 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.935090 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" Jul 31 22:00:47.939181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.939155 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxbg4\" (UniqueName: \"kubernetes.io/projected/91e66e36-c614-401b-b57d-3cfd4257849e-kube-api-access-bxbg4\") pod \"downloads-77c974ffcf-46znc\" (UID: \"91e66e36-c614-401b-b57d-3cfd4257849e\") " pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:00:47.943640 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:47.943502 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:48.007101 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.006851 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fc1f2e38-6147-48be-809e-2ab089aefd8e-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-86rkq\" (UID: \"fc1f2e38-6147-48be-809e-2ab089aefd8e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:48.033100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.033036 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:00:48.102674 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.102640 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65"] Jul 31 22:00:48.104490 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:48.104441 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod472a3167_50e0_424a_8dc9_2fd90c55fc57.slice/crio-1ed7262890d8fc57cb8dabecada82cd6b0093dbefe524cebbae8adcbd80896f2 WatchSource:0}: Error finding container 1ed7262890d8fc57cb8dabecada82cd6b0093dbefe524cebbae8adcbd80896f2: Status 404 returned error can't find the container with id 1ed7262890d8fc57cb8dabecada82cd6b0093dbefe524cebbae8adcbd80896f2 Jul 31 22:00:48.108321 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.107788 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fc1f2e38-6147-48be-809e-2ab089aefd8e-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-86rkq\" (UID: \"fc1f2e38-6147-48be-809e-2ab089aefd8e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:48.113028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.112757 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fc1f2e38-6147-48be-809e-2ab089aefd8e-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-86rkq\" (UID: \"fc1f2e38-6147-48be-809e-2ab089aefd8e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:48.113028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.112911 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:48.123158 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.123132 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj"] Jul 31 22:00:48.125375 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:48.125351 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06053773_ff75_413f_8aec_8f7e2a369f88.slice/crio-a7a5b0dc33e046ed14664c2351ca747f08718028203128166f1267d4d39581d8 WatchSource:0}: Error finding container a7a5b0dc33e046ed14664c2351ca747f08718028203128166f1267d4d39581d8: Status 404 returned error can't find the container with id a7a5b0dc33e046ed14664c2351ca747f08718028203128166f1267d4d39581d8 Jul 31 22:00:48.145698 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.145666 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6"] Jul 31 22:00:48.150021 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:48.149989 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8008fbb7_3f9d_4334_a1f2_9f4de05e9dd3.slice/crio-9242d9bfed902584de206a7c6834dc3495361393320e02feeab0a63156f426c7 WatchSource:0}: Error finding container 9242d9bfed902584de206a7c6834dc3495361393320e02feeab0a63156f426c7: Status 404 returned error can't find the container with id 9242d9bfed902584de206a7c6834dc3495361393320e02feeab0a63156f426c7 Jul 31 22:00:48.183206 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.183177 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:48.204657 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.204618 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-46znc"] Jul 31 22:00:48.208387 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:48.208353 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e66e36_c614_401b_b57d_3cfd4257849e.slice/crio-a227db8632462a80de15aa3c9cf9de3e69c9e664a040657eb0a632c06af0103c WatchSource:0}: Error finding container a227db8632462a80de15aa3c9cf9de3e69c9e664a040657eb0a632c06af0103c: Status 404 returned error can't find the container with id a227db8632462a80de15aa3c9cf9de3e69c9e664a040657eb0a632c06af0103c Jul 31 22:00:48.223194 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.223160 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:48.328061 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.328015 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq"] Jul 31 22:00:48.330506 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.330473 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:48.330946 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:48.330924 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1f2e38_6147_48be_809e_2ab089aefd8e.slice/crio-386ecc51f066e5af656fb5d9a5bc3f814b55f98f97ce05c86b1b3555368a022a WatchSource:0}: Error finding container 386ecc51f066e5af656fb5d9a5bc3f814b55f98f97ce05c86b1b3555368a022a: Status 404 returned error can't find the container with id 386ecc51f066e5af656fb5d9a5bc3f814b55f98f97ce05c86b1b3555368a022a Jul 31 22:00:48.372366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.372252 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" event={"ID":"fc1f2e38-6147-48be-809e-2ab089aefd8e","Type":"ContainerStarted","Data":"386ecc51f066e5af656fb5d9a5bc3f814b55f98f97ce05c86b1b3555368a022a"} Jul 31 22:00:48.373981 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.373949 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" event={"ID":"472a3167-50e0-424a-8dc9-2fd90c55fc57","Type":"ContainerStarted","Data":"1ed7262890d8fc57cb8dabecada82cd6b0093dbefe524cebbae8adcbd80896f2"} Jul 31 22:00:48.375064 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.375028 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-46znc" event={"ID":"91e66e36-c614-401b-b57d-3cfd4257849e","Type":"ContainerStarted","Data":"a227db8632462a80de15aa3c9cf9de3e69c9e664a040657eb0a632c06af0103c"} Jul 31 22:00:48.376241 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.376216 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" event={"ID":"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3","Type":"ContainerStarted","Data":"9242d9bfed902584de206a7c6834dc3495361393320e02feeab0a63156f426c7"} Jul 31 22:00:48.377363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.377338 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" event={"ID":"06053773-ff75-413f-8aec-8f7e2a369f88","Type":"ContainerStarted","Data":"a7a5b0dc33e046ed14664c2351ca747f08718028203128166f1267d4d39581d8"} Jul 31 22:00:48.635695 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.635590 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:48.719965 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.719933 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:48.892378 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:48.892270 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:49.174769 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:49.174580 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:49.242093 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:49.242048 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:50.395624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:50.395111 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" event={"ID":"fc1f2e38-6147-48be-809e-2ab089aefd8e","Type":"ContainerStarted","Data":"5d4610e08fa58b92fe3f994fece511c9fcca494c8628d89c2047c94a1de03b0c"} Jul 31 22:00:50.398034 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:50.396389 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:50.403577 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:50.403530 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" Jul 31 22:00:50.444561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:50.443619 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-86rkq" podStartSLOduration=2.297193345 podStartE2EDuration="3.443582082s" totalImagesPullingTime="1.146388737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:50.422415055 +0000 UTC m=+74.987263641" watchObservedRunningTime="2026-07-31 22:00:50.443582082 +0000 UTC m=+75.008430664" Jul 31 22:00:50.846623 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:50.846573 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:50.933911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:50.933875 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:51.224264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:51.224012 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:52.240221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:52.240193 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8nw5p" Jul 31 22:00:53.220930 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:53.220878 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:53.281555 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:53.281475 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:54.189096 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:54.189059 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:56.149048 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.149004 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:56.468761 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.468698 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:56.554743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.553159 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2zbn5"] Jul 31 22:00:56.556403 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.556376 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.562228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.560476 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 31 22:00:56.562228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.560884 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 31 22:00:56.562228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.561006 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 31 22:00:56.562228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.561078 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c8br7\"" Jul 31 22:00:56.562228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.561211 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586734 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhfm\" (UniqueName: \"kubernetes.io/projected/3293c148-9e45-49dd-924e-3f899cb51143-kube-api-access-5lhfm\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586776 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-textfile\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586813 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-sys\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3293c148-9e45-49dd-924e-3f899cb51143-metrics-client-ca\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586882 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-tls\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586925 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-accelerators-collector-config\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586970 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-wtmp\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.586996 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-root\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.587425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.587042 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.612107 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.612075 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:56.688056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688012 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-tls\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688069 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-accelerators-collector-config\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688109 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-wtmp\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688138 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-root\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688168 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688213 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhfm\" (UniqueName: \"kubernetes.io/projected/3293c148-9e45-49dd-924e-3f899cb51143-kube-api-access-5lhfm\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688488 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688238 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-textfile\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688488 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-sys\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688488 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688301 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3293c148-9e45-49dd-924e-3f899cb51143-metrics-client-ca\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.688488 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:56.688338 2554 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 31 22:00:56.688488 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:00:56.688415 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-tls podName:3293c148-9e45-49dd-924e-3f899cb51143 nodeName:}" failed. No retries permitted until 2026-07-31 22:00:57.188392193 +0000 UTC m=+81.753240765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-tls") pod "node-exporter-2zbn5" (UID: "3293c148-9e45-49dd-924e-3f899cb51143") : secret "node-exporter-tls" not found Jul 31 22:00:56.689818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688895 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-wtmp\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.689818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.688926 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3293c148-9e45-49dd-924e-3f899cb51143-metrics-client-ca\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.689818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.689009 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-root\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.689818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.689207 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3293c148-9e45-49dd-924e-3f899cb51143-sys\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.689818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.689250 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-textfile\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.689818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.689702 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-accelerators-collector-config\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.690422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.690398 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:56.703668 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.702695 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.715325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.711151 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhfm\" (UniqueName: \"kubernetes.io/projected/3293c148-9e45-49dd-924e-3f899cb51143-kube-api-access-5lhfm\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:56.791668 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.791052 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:56.894097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:56.894056 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:57.056960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:57.056285 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:57.193835 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:57.193799 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-tls\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:57.197505 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:57.197422 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3293c148-9e45-49dd-924e-3f899cb51143-node-exporter-tls\") pod \"node-exporter-2zbn5\" (UID: \"3293c148-9e45-49dd-924e-3f899cb51143\") " pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:57.379761 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:57.379693 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4466l" Jul 31 22:00:57.476938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:57.476906 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2zbn5" Jul 31 22:00:57.502184 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:00:57.502142 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3293c148_9e45_49dd_924e_3f899cb51143.slice/crio-5796124d3fdd2e719672d108434dc8aacd4c4e9ddd7871639ecb6b2a2a77470f WatchSource:0}: Error finding container 5796124d3fdd2e719672d108434dc8aacd4c4e9ddd7871639ecb6b2a2a77470f: Status 404 returned error can't find the container with id 5796124d3fdd2e719672d108434dc8aacd4c4e9ddd7871639ecb6b2a2a77470f Jul 31 22:00:57.792542 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:57.792482 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:58.041822 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.041780 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:58.205153 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.204970 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:58.306793 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.306717 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:58.449913 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.449824 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zbn5" event={"ID":"3293c148-9e45-49dd-924e-3f899cb51143","Type":"ContainerStarted","Data":"5796124d3fdd2e719672d108434dc8aacd4c4e9ddd7871639ecb6b2a2a77470f"} Jul 31 22:00:58.453155 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.452874 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" event={"ID":"472a3167-50e0-424a-8dc9-2fd90c55fc57","Type":"ContainerStarted","Data":"d3581be2c2219946528460a66741084e84ce274de82b190c5b3ba8099ad2b2ab"} Jul 31 22:00:58.453155 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.452919 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" event={"ID":"472a3167-50e0-424a-8dc9-2fd90c55fc57","Type":"ContainerStarted","Data":"d17f288acf3ad170ee663e0ae1bd39226b320460d7c3076bb5cb8e6c19e6b7e5"} Jul 31 22:00:58.453155 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.452938 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" event={"ID":"472a3167-50e0-424a-8dc9-2fd90c55fc57","Type":"ContainerStarted","Data":"c83b969ffb91f8c6ceb1021311685f4340865017a756478d4f604506c53b92cf"} Jul 31 22:00:58.454978 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.454840 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" event={"ID":"8008fbb7-3f9d-4334-a1f2-9f4de05e9dd3","Type":"ContainerStarted","Data":"7d8f356147720265907061b8dc915e4ea811e5a8e706aac4ed443524052815ff"} Jul 31 22:00:58.457213 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.457153 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" event={"ID":"06053773-ff75-413f-8aec-8f7e2a369f88","Type":"ContainerStarted","Data":"3238344a9d7cc4bbb318c32a7b0450cd6da67207fcf8362ac5823d749c125335"} Jul 31 22:00:58.457451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.457430 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:58.459298 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.459261 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" Jul 31 22:00:58.477617 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.477497 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c95cd4f6b-4gp65" podStartSLOduration=2.23164605 podStartE2EDuration="11.477478389s" totalImagesPullingTime="9.245832339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:58.474801813 +0000 UTC m=+83.039650395" watchObservedRunningTime="2026-07-31 22:00:58.477478389 +0000 UTC m=+83.042326973" Jul 31 22:00:58.507352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.507299 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-758978cbdd-ghzlj" podStartSLOduration=2.28574966 podStartE2EDuration="11.507281122s" totalImagesPullingTime="9.221531462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:58.502890493 +0000 UTC m=+83.067739075" watchObservedRunningTime="2026-07-31 22:00:58.507281122 +0000 UTC m=+83.072129706" Jul 31 22:00:58.540410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:58.540355 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7554888dcd-rfbg6" podStartSLOduration=2.371019637 podStartE2EDuration="11.540339649s" totalImagesPullingTime="9.169320012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:00:58.537895844 +0000 UTC m=+83.102744434" watchObservedRunningTime="2026-07-31 22:00:58.540339649 +0000 UTC m=+83.105188231" Jul 31 22:00:59.440225 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:59.440189 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:59.463567 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:59.463539 2554 generic.go:356] "Generic (PLEG): container finished" podID="3293c148-9e45-49dd-924e-3f899cb51143" containerID="ff0ecb4d711575e3de0524d3121b6e09884b0229ad7e10be79bacaa58beeb58d" exitCode=0 Jul 31 22:00:59.463745 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:59.463617 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zbn5" event={"ID":"3293c148-9e45-49dd-924e-3f899cb51143","Type":"ContainerDied","Data":"ff0ecb4d711575e3de0524d3121b6e09884b0229ad7e10be79bacaa58beeb58d"} Jul 31 22:00:59.516681 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:59.516469 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:00:59.614792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:00:59.614765 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:00.470979 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:00.470934 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zbn5" event={"ID":"3293c148-9e45-49dd-924e-3f899cb51143","Type":"ContainerStarted","Data":"de2188eb64740da4aa907a3e63f5f463db740f2892413f95e844b051726db141"} Jul 31 22:01:00.471445 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:00.471005 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zbn5" event={"ID":"3293c148-9e45-49dd-924e-3f899cb51143","Type":"ContainerStarted","Data":"32b14d1e63142c93aac82f2ca7c967a58d7d5d73f6d0b7ff0059a4db54096fe3"} Jul 31 22:01:00.494766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:00.494703 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2zbn5" podStartSLOduration=3.196988722 podStartE2EDuration="4.494681501s" totalImagesPullingTime="1.220841497s" totalInitContainerRuntime="76.851282ms" isStatefulPod=true podCreationTimestamp="2026-07-31 22:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:01:00.494226426 +0000 UTC m=+85.059075000" watchObservedRunningTime="2026-07-31 22:01:00.494681501 +0000 UTC m=+85.059530086" Jul 31 22:01:00.775721 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:00.775637 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:00.854499 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:00.854452 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:01.094795 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.094707 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:01.175817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.175781 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:01.308198 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.308123 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:01.408559 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.408437 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:01.459751 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.459709 2554 patch_prober.go:28] interesting pod/image-registry-54bbc9d58-vq54h container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 31 22:01:01.459922 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.459781 2554 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" podUID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:01:01.512466 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:01.512409 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:02.884135 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:02.884099 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:03.347559 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:03.347529 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:01:03.598624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:03.598520 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:03.666693 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:03.666647 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:04.053176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:04.053037 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:04.154686 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:04.154646 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:06.192234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:06.192178 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:06.281276 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:06.281240 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:06.555256 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:06.555170 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:06.638780 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:06.638747 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:08.511962 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:08.511924 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-46znc" event={"ID":"91e66e36-c614-401b-b57d-3cfd4257849e","Type":"ContainerStarted","Data":"f47941e204e85a6389cd628cf8bb63425dbbacbf7790df5e6b26db876b57d897"} Jul 31 22:01:08.512435 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:08.512187 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:01:08.534894 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:08.534617 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-46znc" podStartSLOduration=2.285448729 podStartE2EDuration="21.534583485s" totalImagesPullingTime="19.249134756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:01:08.533854602 +0000 UTC m=+93.098703186" watchObservedRunningTime="2026-07-31 22:01:08.534583485 +0000 UTC m=+93.099432067" Jul 31 22:01:08.535742 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:08.535720 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-46znc" Jul 31 22:01:09.889543 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:09.889501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:09.986967 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:09.986929 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:10.412298 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.412256 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:10.480710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.480671 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:10.603823 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.603788 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54bbc9d58-vq54h"] Jul 31 22:01:10.693487 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.693455 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:10.753769 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.753730 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:10.828630 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.828582 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:10.944041 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:10.943950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:12.268157 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:12.268115 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:12.368861 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:12.368828 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:12.468814 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:12.468777 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:13.156162 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:13.156125 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:13.256802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:13.256766 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:15.040608 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:15.040573 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:15.143155 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:15.143116 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:15.555919 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:15.555878 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:15.656174 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:15.656132 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:17.183354 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:17.183305 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:17.285040 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:17.284981 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:17.451849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:17.451760 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:19.531230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:19.531191 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:19.621889 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:19.621840 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:19.724651 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:19.724594 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:19.824433 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:19.824346 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:24.573935 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:24.573892 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:24.646432 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:24.646397 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:31.339207 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:31.339166 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:31.398518 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:31.398489 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:35.627753 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:35.627708 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" podUID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" containerName="registry" containerID="cri-o://469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d" gracePeriod=30 Jul 31 22:01:35.897862 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:35.897793 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:01:36.034660 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.034623 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.034828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.034718 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-bound-sa-token\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.034828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.034760 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdhzw\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-kube-api-access-kdhzw\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.034828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.034793 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-ca-trust-extracted\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.034968 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.034943 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-certificates\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.035024 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.035001 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-image-registry-private-configuration\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.035078 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.035029 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-trusted-ca\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.035146 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.035086 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-installation-pull-secrets\") pod \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\" (UID: \"5ce1f101-780d-4c40-b7ce-6d9d7844efc8\") " Jul 31 22:01:36.035643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.035307 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:01:36.035643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.035440 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-certificates\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.035838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.035789 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:01:36.037314 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.037287 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:01:36.037439 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.037364 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-kube-api-access-kdhzw" (OuterVolumeSpecName: "kube-api-access-kdhzw") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "kube-api-access-kdhzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:01:36.037517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.037492 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:01:36.037573 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.037538 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:01:36.037762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.037743 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:01:36.043183 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.043154 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5ce1f101-780d-4c40-b7ce-6d9d7844efc8" (UID: "5ce1f101-780d-4c40-b7ce-6d9d7844efc8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:01:36.136407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136377 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdhzw\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-kube-api-access-kdhzw\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.136407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136401 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-ca-trust-extracted\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.136407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136413 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-image-registry-private-configuration\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.136631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136422 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-trusted-ca\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.136631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136434 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-installation-pull-secrets\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.136631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136443 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-registry-tls\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.136631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.136451 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ce1f101-780d-4c40-b7ce-6d9d7844efc8-bound-sa-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:01:36.641232 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.641196 2554 generic.go:356] "Generic (PLEG): container finished" podID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" containerID="469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d" exitCode=0 Jul 31 22:01:36.641714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.641238 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" Jul 31 22:01:36.641714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.641281 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" event={"ID":"5ce1f101-780d-4c40-b7ce-6d9d7844efc8","Type":"ContainerDied","Data":"469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d"} Jul 31 22:01:36.641714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.641331 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54bbc9d58-vq54h" event={"ID":"5ce1f101-780d-4c40-b7ce-6d9d7844efc8","Type":"ContainerDied","Data":"c149bfcc387c1c187019459129f87b6f95fc25b5a3ad6aadb699db517d45ce38"} Jul 31 22:01:36.641714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.641358 2554 scope.go:117] "RemoveContainer" containerID="469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d" Jul 31 22:01:36.642980 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.642961 2554 generic.go:356] "Generic (PLEG): container finished" podID="a19c41c2-668e-495f-aa3f-e9493d1e94c7" containerID="442190d7793af83a18dcf3aac1d93f17c383876db4fc6a326829a52b98f7f9a2" exitCode=0 Jul 31 22:01:36.643071 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.643009 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" event={"ID":"a19c41c2-668e-495f-aa3f-e9493d1e94c7","Type":"ContainerDied","Data":"442190d7793af83a18dcf3aac1d93f17c383876db4fc6a326829a52b98f7f9a2"} Jul 31 22:01:36.643375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.643358 2554 scope.go:117] "RemoveContainer" containerID="442190d7793af83a18dcf3aac1d93f17c383876db4fc6a326829a52b98f7f9a2" Jul 31 22:01:36.652498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.652482 2554 scope.go:117] "RemoveContainer" containerID="469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d" Jul 31 22:01:36.652765 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:01:36.652746 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d\": container with ID starting with 469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d not found: ID does not exist" containerID="469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d" Jul 31 22:01:36.652830 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.652772 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d"} err="failed to get container status \"469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d\": rpc error: code = NotFound desc = could not find container \"469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d\": container with ID starting with 469158c9edbb5bf31bdf0217ff78625726c774a79176324147864c440cf1c12d not found: ID does not exist" Jul 31 22:01:36.683376 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.683343 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54bbc9d58-vq54h"] Jul 31 22:01:36.684744 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:36.684722 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54bbc9d58-vq54h"] Jul 31 22:01:37.650260 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:37.650226 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-8g46m" event={"ID":"a19c41c2-668e-495f-aa3f-e9493d1e94c7","Type":"ContainerStarted","Data":"aab7064ada13200d087885d47fe09b61d665ccafd4cfae040c68b2b2e14f8d5f"} Jul 31 22:01:37.956521 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:37.956485 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" path="/var/lib/kubelet/pods/5ce1f101-780d-4c40-b7ce-6d9d7844efc8/volumes" Jul 31 22:01:41.668049 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:41.667971 2554 generic.go:356] "Generic (PLEG): container finished" podID="502ad9e6-5e67-417b-9efb-6e1625647fd1" containerID="c5ee9bf659fcdb33cbbd46dd366d9a8100870f2325e136904af4ae7db8ed3c55" exitCode=0 Jul 31 22:01:41.668401 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:41.668048 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" event={"ID":"502ad9e6-5e67-417b-9efb-6e1625647fd1","Type":"ContainerDied","Data":"c5ee9bf659fcdb33cbbd46dd366d9a8100870f2325e136904af4ae7db8ed3c55"} Jul 31 22:01:41.668401 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:41.668388 2554 scope.go:117] "RemoveContainer" containerID="c5ee9bf659fcdb33cbbd46dd366d9a8100870f2325e136904af4ae7db8ed3c55" Jul 31 22:01:42.674181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:42.674138 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-6lb44" event={"ID":"502ad9e6-5e67-417b-9efb-6e1625647fd1","Type":"ContainerStarted","Data":"e76b0e66af5d490731cdc80a08f240f0d25af5dbd0b743c4ca92192f2e7d14e6"} Jul 31 22:01:43.844294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:43.844258 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:43.946112 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:43.946071 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:49.675832 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:49.675794 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:49.738719 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:49.738686 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:01:50.706223 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:50.706189 2554 generic.go:356] "Generic (PLEG): container finished" podID="5b3d2cae-4201-4a92-abab-c15ff7865fbf" containerID="2a07a9f7b5f405346f4477af028a1c39ed8c54bbc18ab6062d11d8f36eb7d592" exitCode=0 Jul 31 22:01:50.706630 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:50.706265 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-m67tr" event={"ID":"5b3d2cae-4201-4a92-abab-c15ff7865fbf","Type":"ContainerDied","Data":"2a07a9f7b5f405346f4477af028a1c39ed8c54bbc18ab6062d11d8f36eb7d592"} Jul 31 22:01:50.706630 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:50.706564 2554 scope.go:117] "RemoveContainer" containerID="2a07a9f7b5f405346f4477af028a1c39ed8c54bbc18ab6062d11d8f36eb7d592" Jul 31 22:01:51.712108 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:01:51.712080 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-m67tr" event={"ID":"5b3d2cae-4201-4a92-abab-c15ff7865fbf","Type":"ContainerStarted","Data":"855d4c91e8a8681d3c8565d6925675ff6bbedfe3d85db7947b2a0bccbf9106ae"} Jul 31 22:02:12.597435 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:12.597392 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:12.655280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:12.655232 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:19.424277 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:19.424239 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:19.523809 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:19.523768 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:19.624732 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:19.624699 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:19.690022 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:19.689982 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:20.160458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:20.160369 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:24.418886 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:24.418852 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:24.519894 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:24.519840 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:30.419831 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:30.419748 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:30.524065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:30.524028 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:30.720560 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:30.720528 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:34.684646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:34.684611 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:02:34.782226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:02:34.782192 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:21.376349 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:21.376311 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:21.431887 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:21.431826 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:26.342572 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:26.342531 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:31.072977 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:31.072937 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:31.167144 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:31.167101 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:53.104869 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:53.104830 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:54.110503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:54.110467 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:56.628771 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:56.628734 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:56.730256 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:56.730218 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:57.998774 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:57.998745 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:58.132405 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:58.132366 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:58.813171 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:58.813119 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:03:58.911323 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:03:58.911290 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:00.823557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:00.823513 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:01.300470 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:01.300432 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:03.414398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:03.414365 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:03.514182 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:03.514148 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:07.226281 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:07.226241 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:07.356960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:07.356926 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:09.452980 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:09.452936 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:09.550508 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:09.550472 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:16.141317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:16.141277 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:16.242229 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:16.242193 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:18.044352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:18.044311 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:30.038310 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:30.038261 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:30.178722 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:30.178684 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:30.653402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:30.653325 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:30.852080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:30.852047 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:30.953934 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:30.953907 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:31.011704 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.011676 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr"] Jul 31 22:04:31.011979 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.011967 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" containerName="registry" Jul 31 22:04:31.012027 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.011980 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" containerName="registry" Jul 31 22:04:31.012066 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.012036 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce1f101-780d-4c40-b7ce-6d9d7844efc8" containerName="registry" Jul 31 22:04:31.013803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.013789 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.017098 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.017080 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 31 22:04:31.017480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.017462 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 31 22:04:31.017763 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.017749 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 31 22:04:31.018281 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.018238 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 31 22:04:31.018398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.018378 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wczhl\"" Jul 31 22:04:31.043014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.042980 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr"] Jul 31 22:04:31.105116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.105074 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gw9r\" (UniqueName: \"kubernetes.io/projected/688d7818-2128-46e4-a0ca-8d4f3c4ec619-kube-api-access-6gw9r\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.105116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.105123 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/688d7818-2128-46e4-a0ca-8d4f3c4ec619-manifests\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.105317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.105213 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/688d7818-2128-46e4-a0ca-8d4f3c4ec619-tmp\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.105317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.105254 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/688d7818-2128-46e4-a0ca-8d4f3c4ec619-webhook-cert\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.105317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.105283 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/688d7818-2128-46e4-a0ca-8d4f3c4ec619-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.205189 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.205110 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:31.205919 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.205901 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/688d7818-2128-46e4-a0ca-8d4f3c4ec619-tmp\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.205977 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.205936 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/688d7818-2128-46e4-a0ca-8d4f3c4ec619-webhook-cert\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.205977 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.205962 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/688d7818-2128-46e4-a0ca-8d4f3c4ec619-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.206056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.205993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gw9r\" (UniqueName: \"kubernetes.io/projected/688d7818-2128-46e4-a0ca-8d4f3c4ec619-kube-api-access-6gw9r\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.206056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.206034 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/688d7818-2128-46e4-a0ca-8d4f3c4ec619-manifests\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.206413 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.206392 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/688d7818-2128-46e4-a0ca-8d4f3c4ec619-manifests\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.208300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.208278 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/688d7818-2128-46e4-a0ca-8d4f3c4ec619-tmp\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.208515 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.208495 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/688d7818-2128-46e4-a0ca-8d4f3c4ec619-webhook-cert\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.208557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.208501 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/688d7818-2128-46e4-a0ca-8d4f3c4ec619-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.217138 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.217106 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gw9r\" (UniqueName: \"kubernetes.io/projected/688d7818-2128-46e4-a0ca-8d4f3c4ec619-kube-api-access-6gw9r\") pod \"opendatahub-operator-controller-manager-79bdc6db47-hgzgr\" (UID: \"688d7818-2128-46e4-a0ca-8d4f3c4ec619\") " pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.315951 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.315901 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:31.323870 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.323843 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:31.418165 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.418134 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:31.469844 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.469816 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr"] Jul 31 22:04:31.472432 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:04:31.472405 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod688d7818_2128_46e4_a0ca_8d4f3c4ec619.slice/crio-f2047f9bf8aaf9925cc0b54b33b34105b9245ae167a018a938a838d7e7078251 WatchSource:0}: Error finding container f2047f9bf8aaf9925cc0b54b33b34105b9245ae167a018a938a838d7e7078251: Status 404 returned error can't find the container with id f2047f9bf8aaf9925cc0b54b33b34105b9245ae167a018a938a838d7e7078251 Jul 31 22:04:31.516455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.516426 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:31.620555 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.620523 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:31.723504 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:31.723421 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:32.324014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:32.323980 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:32.334525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:32.334488 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" event={"ID":"688d7818-2128-46e4-a0ca-8d4f3c4ec619","Type":"ContainerStarted","Data":"f2047f9bf8aaf9925cc0b54b33b34105b9245ae167a018a938a838d7e7078251"} Jul 31 22:04:32.426912 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:32.426620 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:35.350723 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:35.350683 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" event={"ID":"688d7818-2128-46e4-a0ca-8d4f3c4ec619","Type":"ContainerStarted","Data":"e88680bc7f7e942364ecc66654d739c322d7bc31eb0813a8ff35f6fc9b949694"} Jul 31 22:04:35.862798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:35.862771 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:04:35.867180 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:35.867149 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:04:35.873656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:35.873636 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 31 22:04:36.356614 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:36.356561 2554 generic.go:356] "Generic (PLEG): container finished" podID="688d7818-2128-46e4-a0ca-8d4f3c4ec619" containerID="e88680bc7f7e942364ecc66654d739c322d7bc31eb0813a8ff35f6fc9b949694" exitCode=0 Jul 31 22:04:36.357913 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:36.356649 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" event={"ID":"688d7818-2128-46e4-a0ca-8d4f3c4ec619","Type":"ContainerDied","Data":"e88680bc7f7e942364ecc66654d739c322d7bc31eb0813a8ff35f6fc9b949694"} Jul 31 22:04:37.362990 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:37.362958 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" event={"ID":"688d7818-2128-46e4-a0ca-8d4f3c4ec619","Type":"ContainerStarted","Data":"6529402ebddfe53e6ca09f768efcf8ae8d5d9149ebda10d6d83f84280df36c5f"} Jul 31 22:04:37.363374 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:37.363112 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:37.390666 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:37.390614 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" podStartSLOduration=3.261593748 podStartE2EDuration="7.390585448s" totalImagesPullingTime="3.106985489s" totalInitContainerRuntime="1.022006211s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:04:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:04:37.389061752 +0000 UTC m=+301.953910331" watchObservedRunningTime="2026-07-31 22:04:37.390585448 +0000 UTC m=+301.955434040" Jul 31 22:04:48.369655 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:48.369622 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79bdc6db47-hgzgr" Jul 31 22:04:50.652903 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:50.652865 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:52.123486 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.123446 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:52.837987 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.837954 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mq52q"] Jul 31 22:04:52.847636 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.847611 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:52.853509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.853482 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 31 22:04:52.883253 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.883218 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mq52q"] Jul 31 22:04:52.887230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.887198 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:52.986591 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.986556 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9nw6\" (UniqueName: \"kubernetes.io/projected/e8108e4d-cfa1-4504-b680-19f86559bc4d-kube-api-access-k9nw6\") pod \"model-serving-api-54d89bd5fc-mq52q\" (UID: \"e8108e4d-cfa1-4504-b680-19f86559bc4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:52.986786 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.986626 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8108e4d-cfa1-4504-b680-19f86559bc4d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mq52q\" (UID: \"e8108e4d-cfa1-4504-b680-19f86559bc4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:52.999421 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:52.999395 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:04:53.088073 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.087993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9nw6\" (UniqueName: \"kubernetes.io/projected/e8108e4d-cfa1-4504-b680-19f86559bc4d-kube-api-access-k9nw6\") pod \"model-serving-api-54d89bd5fc-mq52q\" (UID: \"e8108e4d-cfa1-4504-b680-19f86559bc4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:53.088073 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.088050 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8108e4d-cfa1-4504-b680-19f86559bc4d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mq52q\" (UID: \"e8108e4d-cfa1-4504-b680-19f86559bc4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:53.090785 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.090755 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8108e4d-cfa1-4504-b680-19f86559bc4d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mq52q\" (UID: \"e8108e4d-cfa1-4504-b680-19f86559bc4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:53.096216 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.096173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9nw6\" (UniqueName: \"kubernetes.io/projected/e8108e4d-cfa1-4504-b680-19f86559bc4d-kube-api-access-k9nw6\") pod \"model-serving-api-54d89bd5fc-mq52q\" (UID: \"e8108e4d-cfa1-4504-b680-19f86559bc4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:53.157822 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.157785 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:53.285902 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.285876 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mq52q"] Jul 31 22:04:53.288411 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:04:53.288383 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8108e4d_cfa1_4504_b680_19f86559bc4d.slice/crio-f9c8f4a395188b2706ed421f59af2c3ece596dcd083d134ea1196fd0b7648b1e WatchSource:0}: Error finding container f9c8f4a395188b2706ed421f59af2c3ece596dcd083d134ea1196fd0b7648b1e: Status 404 returned error can't find the container with id f9c8f4a395188b2706ed421f59af2c3ece596dcd083d134ea1196fd0b7648b1e Jul 31 22:04:53.290042 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.290023 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 22:04:53.432015 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:53.431958 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" event={"ID":"e8108e4d-cfa1-4504-b680-19f86559bc4d","Type":"ContainerStarted","Data":"f9c8f4a395188b2706ed421f59af2c3ece596dcd083d134ea1196fd0b7648b1e"} Jul 31 22:04:56.449624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:56.449564 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" event={"ID":"e8108e4d-cfa1-4504-b680-19f86559bc4d","Type":"ContainerStarted","Data":"8baa11eb2a733d3c54b602c448251fe1bbf700ee313b7589fc3c0af445a81b52"} Jul 31 22:04:56.450074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:56.449651 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:04:56.466467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:04:56.466397 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" podStartSLOduration=2.12767071 podStartE2EDuration="4.466382256s" totalImagesPullingTime="2.338711546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:04:56.465325831 +0000 UTC m=+321.030174412" watchObservedRunningTime="2026-07-31 22:04:56.466382256 +0000 UTC m=+321.031230838" Jul 31 22:05:02.449150 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.449112 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:02.512513 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.512481 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l"] Jul 31 22:05:02.515682 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.515665 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.525409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.525388 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 31 22:05:02.525497 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.525409 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 31 22:05:02.543463 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.543435 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l"] Jul 31 22:05:02.664907 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.664874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e3a03e-cfad-4716-89c7-170297098365-tls-certs\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.665052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.664928 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-656cm\" (UniqueName: \"kubernetes.io/projected/40e3a03e-cfad-4716-89c7-170297098365-kube-api-access-656cm\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.665052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.665034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e3a03e-cfad-4716-89c7-170297098365-tmp\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.766541 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.766449 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-656cm\" (UniqueName: \"kubernetes.io/projected/40e3a03e-cfad-4716-89c7-170297098365-kube-api-access-656cm\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.766541 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.766516 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e3a03e-cfad-4716-89c7-170297098365-tmp\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.766541 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.766536 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e3a03e-cfad-4716-89c7-170297098365-tls-certs\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.766854 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:05:02.766654 2554 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 31 22:05:02.766854 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:05:02.766763 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e3a03e-cfad-4716-89c7-170297098365-tls-certs podName:40e3a03e-cfad-4716-89c7-170297098365 nodeName:}" failed. No retries permitted until 2026-07-31 22:05:03.26674102 +0000 UTC m=+327.831589583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/40e3a03e-cfad-4716-89c7-170297098365-tls-certs") pod "kube-auth-proxy-7c56dfbf59-9n74l" (UID: "40e3a03e-cfad-4716-89c7-170297098365") : secret "kube-auth-proxy-tls" not found Jul 31 22:05:02.768817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.768796 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e3a03e-cfad-4716-89c7-170297098365-tmp\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.784201 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.784176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-656cm\" (UniqueName: \"kubernetes.io/projected/40e3a03e-cfad-4716-89c7-170297098365-kube-api-access-656cm\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:02.892073 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.892029 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:02.925581 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.925545 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj"] Jul 31 22:05:02.929139 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.929111 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:02.930503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.930482 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc"] Jul 31 22:05:02.931813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.931793 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-g5vdz\"" Jul 31 22:05:02.932142 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.932126 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 31 22:05:02.933766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.933750 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:02.936132 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.936108 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 31 22:05:02.936407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.936389 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ftlhl\"" Jul 31 22:05:02.949798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.949651 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj"] Jul 31 22:05:02.951926 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:02.951901 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc"] Jul 31 22:05:03.009635 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.008406 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:03.069483 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.069402 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrgqr\" (UniqueName: \"kubernetes.io/projected/da540a84-c3e2-489d-a0d3-4a2815f955d2-kube-api-access-rrgqr\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.069483 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.069437 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrpcd\" (UniqueName: \"kubernetes.io/projected/546e2ef6-5b6d-4baa-86e9-af4af32fb997-kube-api-access-hrpcd\") pod \"kserve-controller-manager-d6f4b65b8-8s5pj\" (UID: \"546e2ef6-5b6d-4baa-86e9-af4af32fb997\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.069483 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.069461 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/546e2ef6-5b6d-4baa-86e9-af4af32fb997-cert\") pod \"kserve-controller-manager-d6f4b65b8-8s5pj\" (UID: \"546e2ef6-5b6d-4baa-86e9-af4af32fb997\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.069735 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.069528 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da540a84-c3e2-489d-a0d3-4a2815f955d2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.094216 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.094177 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:03.170191 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.170160 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrgqr\" (UniqueName: \"kubernetes.io/projected/da540a84-c3e2-489d-a0d3-4a2815f955d2-kube-api-access-rrgqr\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.170191 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.170194 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrpcd\" (UniqueName: \"kubernetes.io/projected/546e2ef6-5b6d-4baa-86e9-af4af32fb997-kube-api-access-hrpcd\") pod \"kserve-controller-manager-d6f4b65b8-8s5pj\" (UID: \"546e2ef6-5b6d-4baa-86e9-af4af32fb997\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.170427 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.170212 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/546e2ef6-5b6d-4baa-86e9-af4af32fb997-cert\") pod \"kserve-controller-manager-d6f4b65b8-8s5pj\" (UID: \"546e2ef6-5b6d-4baa-86e9-af4af32fb997\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.170427 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.170333 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da540a84-c3e2-489d-a0d3-4a2815f955d2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.170535 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:05:03.170467 2554 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 31 22:05:03.170588 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:05:03.170543 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da540a84-c3e2-489d-a0d3-4a2815f955d2-cert podName:da540a84-c3e2-489d-a0d3-4a2815f955d2 nodeName:}" failed. No retries permitted until 2026-07-31 22:05:03.670517023 +0000 UTC m=+328.235365589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/da540a84-c3e2-489d-a0d3-4a2815f955d2-cert") pod "llmisvc-controller-manager-777c6bd7d5-9h5zc" (UID: "da540a84-c3e2-489d-a0d3-4a2815f955d2") : secret "llmisvc-webhook-server-cert" not found Jul 31 22:05:03.172789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.172761 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/546e2ef6-5b6d-4baa-86e9-af4af32fb997-cert\") pod \"kserve-controller-manager-d6f4b65b8-8s5pj\" (UID: \"546e2ef6-5b6d-4baa-86e9-af4af32fb997\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.183152 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.183129 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrgqr\" (UniqueName: \"kubernetes.io/projected/da540a84-c3e2-489d-a0d3-4a2815f955d2-kube-api-access-rrgqr\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.190033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.190016 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrpcd\" (UniqueName: \"kubernetes.io/projected/546e2ef6-5b6d-4baa-86e9-af4af32fb997-kube-api-access-hrpcd\") pod \"kserve-controller-manager-d6f4b65b8-8s5pj\" (UID: \"546e2ef6-5b6d-4baa-86e9-af4af32fb997\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.243080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.243044 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:03.271772 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.271749 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e3a03e-cfad-4716-89c7-170297098365-tls-certs\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:03.273955 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.273933 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e3a03e-cfad-4716-89c7-170297098365-tls-certs\") pod \"kube-auth-proxy-7c56dfbf59-9n74l\" (UID: \"40e3a03e-cfad-4716-89c7-170297098365\") " pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:03.290342 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.290255 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:03.395957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.395930 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj"] Jul 31 22:05:03.398651 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:05:03.398622 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546e2ef6_5b6d_4baa_86e9_af4af32fb997.slice/crio-1a9a82ce2018f61cc307e21efd1575ea414eb513474f1e04d4f4c484765a2be6 WatchSource:0}: Error finding container 1a9a82ce2018f61cc307e21efd1575ea414eb513474f1e04d4f4c484765a2be6: Status 404 returned error can't find the container with id 1a9a82ce2018f61cc307e21efd1575ea414eb513474f1e04d4f4c484765a2be6 Jul 31 22:05:03.424675 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.424649 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" Jul 31 22:05:03.447422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.447381 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:03.483342 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.483312 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" event={"ID":"546e2ef6-5b6d-4baa-86e9-af4af32fb997","Type":"ContainerStarted","Data":"1a9a82ce2018f61cc307e21efd1575ea414eb513474f1e04d4f4c484765a2be6"} Jul 31 22:05:03.559341 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.559305 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l"] Jul 31 22:05:03.561746 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:05:03.561718 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e3a03e_cfad_4716_89c7_170297098365.slice/crio-af1efe8b67050010e37bc4be4c956de9d67a13d6c3ba79490e65e2951674f4bc WatchSource:0}: Error finding container af1efe8b67050010e37bc4be4c956de9d67a13d6c3ba79490e65e2951674f4bc: Status 404 returned error can't find the container with id af1efe8b67050010e37bc4be4c956de9d67a13d6c3ba79490e65e2951674f4bc Jul 31 22:05:03.676321 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.676225 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da540a84-c3e2-489d-a0d3-4a2815f955d2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.678646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.678622 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da540a84-c3e2-489d-a0d3-4a2815f955d2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9h5zc\" (UID: \"da540a84-c3e2-489d-a0d3-4a2815f955d2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:03.853472 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:03.853433 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:04.024254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:04.024218 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc"] Jul 31 22:05:04.026629 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:05:04.026582 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podda540a84_c3e2_489d_a0d3_4a2815f955d2.slice/crio-f10b2c536865236b296b9e4d3d61e175bc465f960f89d9c78b53a26178d2aa2b WatchSource:0}: Error finding container f10b2c536865236b296b9e4d3d61e175bc465f960f89d9c78b53a26178d2aa2b: Status 404 returned error can't find the container with id f10b2c536865236b296b9e4d3d61e175bc465f960f89d9c78b53a26178d2aa2b Jul 31 22:05:04.191990 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:04.191893 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:04.286923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:04.286890 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:04.489074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:04.488984 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" event={"ID":"40e3a03e-cfad-4716-89c7-170297098365","Type":"ContainerStarted","Data":"af1efe8b67050010e37bc4be4c956de9d67a13d6c3ba79490e65e2951674f4bc"} Jul 31 22:05:04.490268 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:04.490240 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" event={"ID":"da540a84-c3e2-489d-a0d3-4a2815f955d2","Type":"ContainerStarted","Data":"f10b2c536865236b296b9e4d3d61e175bc465f960f89d9c78b53a26178d2aa2b"} Jul 31 22:05:05.245350 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:05.245310 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:05.377532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:05.377480 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:06.791726 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:06.791614 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:07.462135 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:07.462108 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mq52q" Jul 31 22:05:07.506674 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:07.506635 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" event={"ID":"40e3a03e-cfad-4716-89c7-170297098365","Type":"ContainerStarted","Data":"6fb5d1e83ef53f6b271ed2bf3ed832b0ef3bdd13aa6b75b7d90b32bd289c92c9"} Jul 31 22:05:07.508285 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:07.508257 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" event={"ID":"546e2ef6-5b6d-4baa-86e9-af4af32fb997","Type":"ContainerStarted","Data":"dbab4c820b13af84fd9866259fde990081b5e0b18a4118f99e9ad1b6a98efa2f"} Jul 31 22:05:07.508406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:07.508373 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:07.525449 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:07.525395 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c56dfbf59-9n74l" podStartSLOduration=2.27388122 podStartE2EDuration="5.525374631s" totalImagesPullingTime="3.251493411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:05:07.523652127 +0000 UTC m=+332.088500710" watchObservedRunningTime="2026-07-31 22:05:07.525374631 +0000 UTC m=+332.090223212" Jul 31 22:05:07.546649 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:07.546576 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" podStartSLOduration=3.108796457 podStartE2EDuration="5.546554811s" totalImagesPullingTime="2.437758354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:05:07.544646683 +0000 UTC m=+332.109495270" watchObservedRunningTime="2026-07-31 22:05:07.546554811 +0000 UTC m=+332.111403395" Jul 31 22:05:08.514571 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:08.514538 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" event={"ID":"da540a84-c3e2-489d-a0d3-4a2815f955d2","Type":"ContainerStarted","Data":"59969acc7b35c20217b7a833fa44384dde1e1a312691945b33123d5c0053a01f"} Jul 31 22:05:08.514961 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:08.514716 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:08.553903 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:08.553848 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" podStartSLOduration=2.157788065 podStartE2EDuration="6.553830008s" totalImagesPullingTime="4.396041943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:05:08.549510545 +0000 UTC m=+333.114359126" watchObservedRunningTime="2026-07-31 22:05:08.553830008 +0000 UTC m=+333.118678622" Jul 31 22:05:09.125451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:09.125401 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:09.336065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:09.336013 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:10.083153 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.083110 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2"] Jul 31 22:05:10.086532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.086515 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.089175 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.089155 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bcw6v\"" Jul 31 22:05:10.089677 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.089660 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 31 22:05:10.101921 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.101898 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2"] Jul 31 22:05:10.115870 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.115848 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:10.143532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143510 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143667 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143536 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143667 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143558 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143667 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143622 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143667 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143660 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143682 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143716 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143745 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lb9r\" (UniqueName: \"kubernetes.io/projected/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-kube-api-access-9lb9r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.143803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.143773 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244480 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244536 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lb9r\" (UniqueName: \"kubernetes.io/projected/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-kube-api-access-9lb9r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244566 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244650 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244674 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244740 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244780 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.244986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.244806 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.245172 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.245148 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.245251 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.245230 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.245312 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.245297 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.245367 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.245353 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.245540 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.245521 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.246884 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.246865 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.247360 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.247343 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.252953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.252927 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:10.258950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.258930 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.259523 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.259503 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lb9r\" (UniqueName: \"kubernetes.io/projected/09b0c9c6-1a5b-43d0-aa14-5d1bb6497817-kube-api-access-9lb9r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2\" (UID: \"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.396676 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.396567 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:10.531799 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.531763 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:10.552894 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:10.552869 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2"] Jul 31 22:05:10.555420 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:05:10.555395 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b0c9c6_1a5b_43d0_aa14_5d1bb6497817.slice/crio-f5548d3573d82523044e93b745475431e8890e59c5936c4d38f9566f8b8f316a WatchSource:0}: Error finding container f5548d3573d82523044e93b745475431e8890e59c5936c4d38f9566f8b8f316a: Status 404 returned error can't find the container with id f5548d3573d82523044e93b745475431e8890e59c5936c4d38f9566f8b8f316a Jul 31 22:05:11.530959 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:11.530921 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" event={"ID":"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817","Type":"ContainerStarted","Data":"f5548d3573d82523044e93b745475431e8890e59c5936c4d38f9566f8b8f316a"} Jul 31 22:05:13.229267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:13.229229 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:05:13.229544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:13.229301 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:05:13.229544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:13.229330 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:05:13.542628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:13.542519 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" event={"ID":"09b0c9c6-1a5b-43d0-aa14-5d1bb6497817","Type":"ContainerStarted","Data":"b8a5754547e4bebeab4b24657c07923a9b1ad63df212fc32ed008b6f1a2fc6b9"} Jul 31 22:05:13.561374 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:13.561315 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" podStartSLOduration=0.889577681 podStartE2EDuration="3.561296856s" totalImagesPullingTime="2.671719175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:05:13.561002638 +0000 UTC m=+338.125851219" watchObservedRunningTime="2026-07-31 22:05:13.561296856 +0000 UTC m=+338.126145441" Jul 31 22:05:14.397694 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:14.397655 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:14.402027 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:14.402000 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:14.546972 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:14.546938 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:14.548022 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:14.547997 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2" Jul 31 22:05:38.519747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:38.519671 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8s5pj" Jul 31 22:05:39.523016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:39.522986 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9h5zc" Jul 31 22:05:52.081846 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:52.081806 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:05:52.177544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:05:52.177494 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:05.821467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:05.821431 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:06.541473 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:06.541440 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:07.346047 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:07.346011 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:09.635747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:09.635716 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:09.742192 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:09.742155 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:10.367144 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.367110 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84"] Jul 31 22:06:10.369822 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.369805 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.372354 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.372332 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2sxtx\"" Jul 31 22:06:10.372562 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.372543 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 31 22:06:10.372617 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.372565 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 31 22:06:10.385709 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.385683 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84"] Jul 31 22:06:10.449007 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.448968 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ee2bf18-c08f-48c2-84db-3e30458761a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.449007 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.449014 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlqtr\" (UniqueName: \"kubernetes.io/projected/0ee2bf18-c08f-48c2-84db-3e30458761a0-kube-api-access-wlqtr\") pod \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.549971 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.549943 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ee2bf18-c08f-48c2-84db-3e30458761a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.550124 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.549987 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlqtr\" (UniqueName: \"kubernetes.io/projected/0ee2bf18-c08f-48c2-84db-3e30458761a0-kube-api-access-wlqtr\") pod \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.555592 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.550865 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ee2bf18-c08f-48c2-84db-3e30458761a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.579364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.579331 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlqtr\" (UniqueName: \"kubernetes.io/projected/0ee2bf18-c08f-48c2-84db-3e30458761a0-kube-api-access-wlqtr\") pod \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.599643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.599590 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:10.679659 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.679567 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:10.811356 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.811270 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84"] Jul 31 22:06:10.813815 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:06:10.813780 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee2bf18_c08f_48c2_84db_3e30458761a0.slice/crio-a9d7a5136938f21a9e2b7b794036ae88769a3310c97674490e28893423ff39ab WatchSource:0}: Error finding container a9d7a5136938f21a9e2b7b794036ae88769a3310c97674490e28893423ff39ab: Status 404 returned error can't find the container with id a9d7a5136938f21a9e2b7b794036ae88769a3310c97674490e28893423ff39ab Jul 31 22:06:10.967855 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:10.967819 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:11.305846 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:11.305757 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:11.801382 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:11.801335 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" event={"ID":"0ee2bf18-c08f-48c2-84db-3e30458761a0","Type":"ContainerStarted","Data":"a9d7a5136938f21a9e2b7b794036ae88769a3310c97674490e28893423ff39ab"} Jul 31 22:06:12.923050 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:12.923010 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-fmrd5"] Jul 31 22:06:12.929723 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:12.929693 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:12.933643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:12.933619 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-h4gkn\"" Jul 31 22:06:12.944134 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:12.944068 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-fmrd5"] Jul 31 22:06:12.968320 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:12.968288 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqf94\" (UniqueName: \"kubernetes.io/projected/7c8c7e6b-3d3c-44f4-8029-72db538db29f-kube-api-access-dqf94\") pod \"authorino-operator-68d88948d8-fmrd5\" (UID: \"7c8c7e6b-3d3c-44f4-8029-72db538db29f\") " pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:13.069532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:13.069492 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqf94\" (UniqueName: \"kubernetes.io/projected/7c8c7e6b-3d3c-44f4-8029-72db538db29f-kube-api-access-dqf94\") pod \"authorino-operator-68d88948d8-fmrd5\" (UID: \"7c8c7e6b-3d3c-44f4-8029-72db538db29f\") " pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:13.079910 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:13.079870 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqf94\" (UniqueName: \"kubernetes.io/projected/7c8c7e6b-3d3c-44f4-8029-72db538db29f-kube-api-access-dqf94\") pod \"authorino-operator-68d88948d8-fmrd5\" (UID: \"7c8c7e6b-3d3c-44f4-8029-72db538db29f\") " pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:13.243662 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:13.243629 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:13.401557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:13.401537 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-fmrd5"] Jul 31 22:06:13.404977 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:06:13.404949 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c8c7e6b_3d3c_44f4_8029_72db538db29f.slice/crio-01586e0af6f81bd07105bb55c2b196e1ca2b2d505cdb096df0a236fbc9940131 WatchSource:0}: Error finding container 01586e0af6f81bd07105bb55c2b196e1ca2b2d505cdb096df0a236fbc9940131: Status 404 returned error can't find the container with id 01586e0af6f81bd07105bb55c2b196e1ca2b2d505cdb096df0a236fbc9940131 Jul 31 22:06:13.708459 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:13.708391 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:13.813920 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:13.813875 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" event={"ID":"7c8c7e6b-3d3c-44f4-8029-72db538db29f","Type":"ContainerStarted","Data":"01586e0af6f81bd07105bb55c2b196e1ca2b2d505cdb096df0a236fbc9940131"} Jul 31 22:06:15.403867 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.403823 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:15.609847 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.609812 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:15.824954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.824915 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" event={"ID":"0ee2bf18-c08f-48c2-84db-3e30458761a0","Type":"ContainerStarted","Data":"4ea894c9a911cf96c9b3b7dd624ea284131dbad04b10800064a6b6a87f64d5e4"} Jul 31 22:06:15.825142 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.825018 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:15.826739 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.826714 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" event={"ID":"7c8c7e6b-3d3c-44f4-8029-72db538db29f","Type":"ContainerStarted","Data":"285e5a9a18a253d2c7ea3c144f62b4c7c434b35b347bc7e2bf1194b594f66440"} Jul 31 22:06:15.826846 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.826822 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:15.848415 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.848367 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" podStartSLOduration=1.7792574810000001 podStartE2EDuration="5.848350878s" totalImagesPullingTime="4.069093397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:06:15.847054788 +0000 UTC m=+400.411903370" watchObservedRunningTime="2026-07-31 22:06:15.848350878 +0000 UTC m=+400.413199462" Jul 31 22:06:15.886712 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:15.886647 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" podStartSLOduration=1.729228828 podStartE2EDuration="3.88662422s" totalImagesPullingTime="2.157395392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:06:15.875193271 +0000 UTC m=+400.440041853" watchObservedRunningTime="2026-07-31 22:06:15.88662422 +0000 UTC m=+400.451472802" Jul 31 22:06:16.001855 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:16.001815 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:16.101645 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:16.101529 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:16.806898 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:16.806859 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:18.029363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:18.029326 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:18.334407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:18.334313 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:20.856967 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:20.856932 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:20.973800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:20.973762 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:21.216929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:21.216899 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:21.363351 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:21.363317 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:21.574756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:21.574667 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:21.674895 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:21.674862 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:25.494647 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:25.494590 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:26.833891 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:26.833825 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-fmrd5" Jul 31 22:06:26.834276 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:26.833930 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:27.032325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:27.032289 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:27.266627 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:27.266568 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:27.380387 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:27.380353 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:28.669718 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.669683 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84"] Jul 31 22:06:28.670162 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.669875 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" containerName="manager" containerID="cri-o://4ea894c9a911cf96c9b3b7dd624ea284131dbad04b10800064a6b6a87f64d5e4" gracePeriod=2 Jul 31 22:06:28.686162 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.686091 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84"] Jul 31 22:06:28.693744 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:06:28.693707 2554 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"0ee2bf18-c08f-48c2-84db-3e30458761a0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-m7h84\": pods \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:28.705932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.705888 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt"] Jul 31 22:06:28.706458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.706438 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" containerName="manager" Jul 31 22:06:28.706458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.706459 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" containerName="manager" Jul 31 22:06:28.706643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.706558 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" containerName="manager" Jul 31 22:06:28.709783 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.709765 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.743804 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.743503 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt"] Jul 31 22:06:28.775282 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:06:28.775237 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" is forbidden: User \"system:node:ip-10-0-140-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-197.ec2.internal' and this object" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:28.807682 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.807632 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/175b4359-a487-42d7-8d86-bd68335eca67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5lblt\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.807828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.807751 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqstl\" (UniqueName: \"kubernetes.io/projected/175b4359-a487-42d7-8d86-bd68335eca67-kube-api-access-wqstl\") pod \"kuadrant-operator-controller-manager-58fc56757c-5lblt\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.896877 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.896846 2554 generic.go:356] "Generic (PLEG): container finished" podID="0ee2bf18-c08f-48c2-84db-3e30458761a0" containerID="4ea894c9a911cf96c9b3b7dd624ea284131dbad04b10800064a6b6a87f64d5e4" exitCode=0 Jul 31 22:06:28.909147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.909123 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/175b4359-a487-42d7-8d86-bd68335eca67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5lblt\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.909270 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.909176 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqstl\" (UniqueName: \"kubernetes.io/projected/175b4359-a487-42d7-8d86-bd68335eca67-kube-api-access-wqstl\") pod \"kuadrant-operator-controller-manager-58fc56757c-5lblt\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.909566 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.909540 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/175b4359-a487-42d7-8d86-bd68335eca67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5lblt\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.923411 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.923364 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:28.924020 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:28.924002 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqstl\" (UniqueName: \"kubernetes.io/projected/175b4359-a487-42d7-8d86-bd68335eca67-kube-api-access-wqstl\") pod \"kuadrant-operator-controller-manager-58fc56757c-5lblt\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:28.925970 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:06:28.925944 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" is forbidden: User \"system:node:ip-10-0-140-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-197.ec2.internal' and this object" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:29.010424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.010388 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlqtr\" (UniqueName: \"kubernetes.io/projected/0ee2bf18-c08f-48c2-84db-3e30458761a0-kube-api-access-wlqtr\") pod \"0ee2bf18-c08f-48c2-84db-3e30458761a0\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " Jul 31 22:06:29.010424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.010426 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ee2bf18-c08f-48c2-84db-3e30458761a0-extensions-socket-volume\") pod \"0ee2bf18-c08f-48c2-84db-3e30458761a0\" (UID: \"0ee2bf18-c08f-48c2-84db-3e30458761a0\") " Jul 31 22:06:29.010932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.010905 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee2bf18-c08f-48c2-84db-3e30458761a0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ee2bf18-c08f-48c2-84db-3e30458761a0" (UID: "0ee2bf18-c08f-48c2-84db-3e30458761a0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:06:29.012572 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.012549 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee2bf18-c08f-48c2-84db-3e30458761a0-kube-api-access-wlqtr" (OuterVolumeSpecName: "kube-api-access-wlqtr") pod "0ee2bf18-c08f-48c2-84db-3e30458761a0" (UID: "0ee2bf18-c08f-48c2-84db-3e30458761a0"). InnerVolumeSpecName "kube-api-access-wlqtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:06:29.048728 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.048699 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:29.111612 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.111559 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlqtr\" (UniqueName: \"kubernetes.io/projected/0ee2bf18-c08f-48c2-84db-3e30458761a0-kube-api-access-wlqtr\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:06:29.111612 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.111587 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ee2bf18-c08f-48c2-84db-3e30458761a0-extensions-socket-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:06:29.184398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.184375 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt"] Jul 31 22:06:29.186902 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:06:29.186873 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175b4359_a487_42d7_8d86_bd68335eca67.slice/crio-74aa466fe2a49433c426d142b5a162074a7f92e6a32dbbc2b738c9c2a594474c WatchSource:0}: Error finding container 74aa466fe2a49433c426d142b5a162074a7f92e6a32dbbc2b738c9c2a594474c: Status 404 returned error can't find the container with id 74aa466fe2a49433c426d142b5a162074a7f92e6a32dbbc2b738c9c2a594474c Jul 31 22:06:29.606146 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.606062 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:29.707948 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.707917 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:29.802563 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.802527 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc"] Jul 31 22:06:29.806077 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.806050 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:29.816024 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:06:29.815999 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" is forbidden: User \"system:node:ip-10-0-140-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-197.ec2.internal' and this object" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:29.817798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.817770 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:29.837542 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.837519 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc"] Jul 31 22:06:29.904430 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.904344 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" event={"ID":"175b4359-a487-42d7-8d86-bd68335eca67","Type":"ContainerStarted","Data":"a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970"} Jul 31 22:06:29.904430 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.904401 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" event={"ID":"175b4359-a487-42d7-8d86-bd68335eca67","Type":"ContainerStarted","Data":"74aa466fe2a49433c426d142b5a162074a7f92e6a32dbbc2b738c9c2a594474c"} Jul 31 22:06:29.904662 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.904501 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:29.905843 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.905821 2554 scope.go:117] "RemoveContainer" containerID="4ea894c9a911cf96c9b3b7dd624ea284131dbad04b10800064a6b6a87f64d5e4" Jul 31 22:06:29.905950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.905823 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:29.919897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.919875 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/336cd904-3767-4dca-903c-39bd5ec13181-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-flgjc\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:29.920002 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.919907 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfpfp\" (UniqueName: \"kubernetes.io/projected/336cd904-3767-4dca-903c-39bd5ec13181-kube-api-access-tfpfp\") pod \"kuadrant-operator-controller-manager-68cf589589-flgjc\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:29.956327 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.956280 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" podStartSLOduration=1.956266869 podStartE2EDuration="1.956266869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:06:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:06:29.95583767 +0000 UTC m=+414.520686246" watchObservedRunningTime="2026-07-31 22:06:29.956266869 +0000 UTC m=+414.521115451" Jul 31 22:06:29.956575 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:29.956553 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" path="/var/lib/kubelet/pods/0ee2bf18-c08f-48c2-84db-3e30458761a0/volumes" Jul 31 22:06:29.960739 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:06:29.960716 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-m7h84\" is forbidden: User \"system:node:ip-10-0-140-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-197.ec2.internal' and this object" podUID="0ee2bf18-c08f-48c2-84db-3e30458761a0" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-m7h84" Jul 31 22:06:30.020518 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.020486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/336cd904-3767-4dca-903c-39bd5ec13181-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-flgjc\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:30.020698 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.020530 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfpfp\" (UniqueName: \"kubernetes.io/projected/336cd904-3767-4dca-903c-39bd5ec13181-kube-api-access-tfpfp\") pod \"kuadrant-operator-controller-manager-68cf589589-flgjc\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:30.020903 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.020882 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/336cd904-3767-4dca-903c-39bd5ec13181-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-flgjc\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:30.035356 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.035334 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfpfp\" (UniqueName: \"kubernetes.io/projected/336cd904-3767-4dca-903c-39bd5ec13181-kube-api-access-tfpfp\") pod \"kuadrant-operator-controller-manager-68cf589589-flgjc\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:30.116069 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.116040 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:30.291957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.291933 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc"] Jul 31 22:06:30.294235 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:06:30.294203 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336cd904_3767_4dca_903c_39bd5ec13181.slice/crio-f7d68f2b9c60509a14e60a47b8b87979b69ce8fd64745715becb841dd11b1638 WatchSource:0}: Error finding container f7d68f2b9c60509a14e60a47b8b87979b69ce8fd64745715becb841dd11b1638: Status 404 returned error can't find the container with id f7d68f2b9c60509a14e60a47b8b87979b69ce8fd64745715becb841dd11b1638 Jul 31 22:06:30.914199 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.914157 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" event={"ID":"336cd904-3767-4dca-903c-39bd5ec13181","Type":"ContainerStarted","Data":"9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad"} Jul 31 22:06:30.914620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.914227 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" event={"ID":"336cd904-3767-4dca-903c-39bd5ec13181","Type":"ContainerStarted","Data":"f7d68f2b9c60509a14e60a47b8b87979b69ce8fd64745715becb841dd11b1638"} Jul 31 22:06:30.914620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:30.914407 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:31.257845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:31.257811 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:31.366261 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:31.366229 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:32.109347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:32.109312 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:32.209500 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:32.209463 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:34.635512 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:34.635447 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:35.643878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:35.643828 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:36.773039 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:36.773003 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:36.873514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:36.873481 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:40.043018 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:40.042984 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:40.916921 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:40.916894 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:40.941838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:40.941786 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" podStartSLOduration=11.941772258 podStartE2EDuration="11.941772258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:06:30.993246027 +0000 UTC m=+415.558094609" watchObservedRunningTime="2026-07-31 22:06:40.941772258 +0000 UTC m=+425.506620840" Jul 31 22:06:41.922511 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:41.922483 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:06:42.028123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.028089 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt"] Jul 31 22:06:42.028376 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.028350 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" podUID="175b4359-a487-42d7-8d86-bd68335eca67" containerName="manager" containerID="cri-o://a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970" gracePeriod=10 Jul 31 22:06:42.042810 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.042774 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:42.139032 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.138995 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:42.308652 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.308623 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:42.431062 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.431027 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/175b4359-a487-42d7-8d86-bd68335eca67-extensions-socket-volume\") pod \"175b4359-a487-42d7-8d86-bd68335eca67\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " Jul 31 22:06:42.431240 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.431159 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqstl\" (UniqueName: \"kubernetes.io/projected/175b4359-a487-42d7-8d86-bd68335eca67-kube-api-access-wqstl\") pod \"175b4359-a487-42d7-8d86-bd68335eca67\" (UID: \"175b4359-a487-42d7-8d86-bd68335eca67\") " Jul 31 22:06:42.431450 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.431430 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/175b4359-a487-42d7-8d86-bd68335eca67-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "175b4359-a487-42d7-8d86-bd68335eca67" (UID: "175b4359-a487-42d7-8d86-bd68335eca67"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:06:42.433234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.433211 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175b4359-a487-42d7-8d86-bd68335eca67-kube-api-access-wqstl" (OuterVolumeSpecName: "kube-api-access-wqstl") pod "175b4359-a487-42d7-8d86-bd68335eca67" (UID: "175b4359-a487-42d7-8d86-bd68335eca67"). InnerVolumeSpecName "kube-api-access-wqstl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:06:42.532452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.532369 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqstl\" (UniqueName: \"kubernetes.io/projected/175b4359-a487-42d7-8d86-bd68335eca67-kube-api-access-wqstl\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:06:42.532452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.532400 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/175b4359-a487-42d7-8d86-bd68335eca67-extensions-socket-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:06:42.550963 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.550930 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:42.728063 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.728010 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:42.835961 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.835878 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:42.929757 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.929717 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:42.974927 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.974899 2554 generic.go:356] "Generic (PLEG): container finished" podID="175b4359-a487-42d7-8d86-bd68335eca67" containerID="a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970" exitCode=0 Jul 31 22:06:42.975097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.974943 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" Jul 31 22:06:42.975097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.974954 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" event={"ID":"175b4359-a487-42d7-8d86-bd68335eca67","Type":"ContainerDied","Data":"a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970"} Jul 31 22:06:42.975097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.974996 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt" event={"ID":"175b4359-a487-42d7-8d86-bd68335eca67","Type":"ContainerDied","Data":"74aa466fe2a49433c426d142b5a162074a7f92e6a32dbbc2b738c9c2a594474c"} Jul 31 22:06:42.975097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.975019 2554 scope.go:117] "RemoveContainer" containerID="a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970" Jul 31 22:06:42.983877 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.983858 2554 scope.go:117] "RemoveContainer" containerID="a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970" Jul 31 22:06:42.984139 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:06:42.984117 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970\": container with ID starting with a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970 not found: ID does not exist" containerID="a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970" Jul 31 22:06:42.984193 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.984147 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970"} err="failed to get container status \"a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970\": rpc error: code = NotFound desc = could not find container \"a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970\": container with ID starting with a10f07cc5e29f2af991f0203a68b0c7ed53130a68a5dc2007fa69b4e3951a970 not found: ID does not exist" Jul 31 22:06:42.999223 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:42.999196 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt"] Jul 31 22:06:43.005490 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:43.005470 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5lblt"] Jul 31 22:06:43.956921 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:43.956886 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="175b4359-a487-42d7-8d86-bd68335eca67" path="/var/lib/kubelet/pods/175b4359-a487-42d7-8d86-bd68335eca67/volumes" Jul 31 22:06:51.384686 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:51.384648 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:51.484942 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:51.484904 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:53.890233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:53.890191 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:53.990362 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:53.990325 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:58.690255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.690173 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:58.725302 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.725270 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc"] Jul 31 22:06:58.725714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.725697 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175b4359-a487-42d7-8d86-bd68335eca67" containerName="manager" Jul 31 22:06:58.725815 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.725717 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="175b4359-a487-42d7-8d86-bd68335eca67" containerName="manager" Jul 31 22:06:58.725891 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.725876 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="175b4359-a487-42d7-8d86-bd68335eca67" containerName="manager" Jul 31 22:06:58.728553 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.728531 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.732819 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.732801 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pjfkr\"" Jul 31 22:06:58.744540 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.744513 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc"] Jul 31 22:06:58.786000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.785963 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:58.870608 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870560 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870782 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870658 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870782 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870701 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c2xr\" (UniqueName: \"kubernetes.io/projected/b8d36c88-b636-43e4-a316-a9101dc08989-kube-api-access-7c2xr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870782 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870740 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8d36c88-b636-43e4-a316-a9101dc08989-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870782 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870761 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8d36c88-b636-43e4-a316-a9101dc08989-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870958 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870843 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8d36c88-b636-43e4-a316-a9101dc08989-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870958 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870908 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.870958 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870946 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.871079 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.870978 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972386 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972288 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972386 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972345 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c2xr\" (UniqueName: \"kubernetes.io/projected/b8d36c88-b636-43e4-a316-a9101dc08989-kube-api-access-7c2xr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8d36c88-b636-43e4-a316-a9101dc08989-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972417 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8d36c88-b636-43e4-a316-a9101dc08989-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972565 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8d36c88-b636-43e4-a316-a9101dc08989-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972640 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972673 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972701 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972774 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.972878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.972782 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.973142 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.973117 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.973265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.973152 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.973265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.973171 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.973265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.973173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8d36c88-b636-43e4-a316-a9101dc08989-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.974866 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.974844 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8d36c88-b636-43e4-a316-a9101dc08989-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.975273 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.975256 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8d36c88-b636-43e4-a316-a9101dc08989-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.980898 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.980873 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c2xr\" (UniqueName: \"kubernetes.io/projected/b8d36c88-b636-43e4-a316-a9101dc08989-kube-api-access-7c2xr\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:58.980999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:58.980876 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8d36c88-b636-43e4-a316-a9101dc08989-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hhkcc\" (UID: \"b8d36c88-b636-43e4-a316-a9101dc08989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:59.042755 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:59.042724 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:06:59.063483 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:59.063449 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:06:59.181842 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:59.181816 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc"] Jul 31 22:06:59.183519 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:06:59.183491 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d36c88_b636_43e4_a316_a9101dc08989.slice/crio-c0c610fdfc18d425391c495c25865f05536b4b56b5384bb69b120f244dd07398 WatchSource:0}: Error finding container c0c610fdfc18d425391c495c25865f05536b4b56b5384bb69b120f244dd07398: Status 404 returned error can't find the container with id c0c610fdfc18d425391c495c25865f05536b4b56b5384bb69b120f244dd07398 Jul 31 22:06:59.185446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:59.185415 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:06:59.185523 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:59.185483 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:06:59.185523 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:06:59.185512 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:07:00.056517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:00.056482 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" event={"ID":"b8d36c88-b636-43e4-a316-a9101dc08989","Type":"ContainerStarted","Data":"73caba1caa76df9e9c2d39ee0374c1a7a450028dbd4ce30419378215b06c1bd2"} Jul 31 22:07:00.056920 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:00.056535 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" event={"ID":"b8d36c88-b636-43e4-a316-a9101dc08989","Type":"ContainerStarted","Data":"c0c610fdfc18d425391c495c25865f05536b4b56b5384bb69b120f244dd07398"} Jul 31 22:07:00.083508 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:00.083453 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" podStartSLOduration=2.083439687 podStartE2EDuration="2.083439687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:07:00.081653283 +0000 UTC m=+444.646501862" watchObservedRunningTime="2026-07-31 22:07:00.083439687 +0000 UTC m=+444.648288268" Jul 31 22:07:01.042906 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.042874 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:07:01.048426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.048402 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:07:01.061458 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.061434 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:07:01.062364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.062341 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:07:01.126557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.126520 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:01.212320 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.212285 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:01.281282 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.281244 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:01.445195 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.445163 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:07:01.448716 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.448695 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.450928 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.450905 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 31 22:07:01.451037 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.450965 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8wdfd\"" Jul 31 22:07:01.459108 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.459085 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:07:01.478012 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.477988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:07:01.597800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.597767 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55ab41b6-21a2-4c19-93cf-2d3414f8f1b9-config-file\") pod \"limitador-limitador-85d9db4d5d-xjr6m\" (UID: \"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.597954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.597891 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdxgz\" (UniqueName: \"kubernetes.io/projected/55ab41b6-21a2-4c19-93cf-2d3414f8f1b9-kube-api-access-sdxgz\") pod \"limitador-limitador-85d9db4d5d-xjr6m\" (UID: \"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.698554 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.698472 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdxgz\" (UniqueName: \"kubernetes.io/projected/55ab41b6-21a2-4c19-93cf-2d3414f8f1b9-kube-api-access-sdxgz\") pod \"limitador-limitador-85d9db4d5d-xjr6m\" (UID: \"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.698554 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.698520 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55ab41b6-21a2-4c19-93cf-2d3414f8f1b9-config-file\") pod \"limitador-limitador-85d9db4d5d-xjr6m\" (UID: \"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.699148 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.699130 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55ab41b6-21a2-4c19-93cf-2d3414f8f1b9-config-file\") pod \"limitador-limitador-85d9db4d5d-xjr6m\" (UID: \"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.707099 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.707073 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdxgz\" (UniqueName: \"kubernetes.io/projected/55ab41b6-21a2-4c19-93cf-2d3414f8f1b9-kube-api-access-sdxgz\") pod \"limitador-limitador-85d9db4d5d-xjr6m\" (UID: \"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.759716 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.759681 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:01.888424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.888359 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:07:01.890939 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:07:01.890907 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ab41b6_21a2_4c19_93cf_2d3414f8f1b9.slice/crio-d996ce5679b82cfd27ee0e56928743dc806ada1fa3d048653e9b6e2614f249da WatchSource:0}: Error finding container d996ce5679b82cfd27ee0e56928743dc806ada1fa3d048653e9b6e2614f249da: Status 404 returned error can't find the container with id d996ce5679b82cfd27ee0e56928743dc806ada1fa3d048653e9b6e2614f249da Jul 31 22:07:01.975222 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:01.975139 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:02.046148 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:02.046111 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:02.067048 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:02.067018 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" event={"ID":"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9","Type":"ContainerStarted","Data":"d996ce5679b82cfd27ee0e56928743dc806ada1fa3d048653e9b6e2614f249da"} Jul 31 22:07:04.078665 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:04.078625 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" event={"ID":"55ab41b6-21a2-4c19-93cf-2d3414f8f1b9","Type":"ContainerStarted","Data":"27005c55dedf6a68cc6a76c10f8b68932ac4251b642ce91c27911773a8cb0e2c"} Jul 31 22:07:04.079047 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:04.078816 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:04.099293 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:04.099248 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" podStartSLOduration=1.848072972 podStartE2EDuration="3.099234279s" totalImagesPullingTime="1.251161307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:07:04.097066537 +0000 UTC m=+448.661915124" watchObservedRunningTime="2026-07-31 22:07:04.099234279 +0000 UTC m=+448.664082859" Jul 31 22:07:15.086233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:15.086205 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m" Jul 31 22:07:17.277899 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:17.277859 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:17.375430 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:17.375384 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:17.739301 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:17.739262 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:27.279845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:27.279806 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:30.048647 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:30.048613 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:30.484964 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:30.484931 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:36.977036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:36.976998 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:37.058278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:37.058240 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:37.121924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:37.121885 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:37.192653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:37.192588 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:39.251353 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:39.251319 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:07:39.348255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:07:39.348225 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:11.954279 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:11.954241 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:12.051788 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:12.051743 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:13.612442 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:13.612408 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:13.838003 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:13.837971 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:14.333856 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:14.333761 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:14.433426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:14.433388 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:14.606452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:14.606372 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:08:23.005743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.005708 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:23.115428 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.115370 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:23.450957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.450929 2554 generic.go:356] "Generic (PLEG): container finished" podID="b8d36c88-b636-43e4-a316-a9101dc08989" containerID="73caba1caa76df9e9c2d39ee0374c1a7a450028dbd4ce30419378215b06c1bd2" exitCode=0 Jul 31 22:08:23.451131 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.451003 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" event={"ID":"b8d36c88-b636-43e4-a316-a9101dc08989","Type":"ContainerDied","Data":"73caba1caa76df9e9c2d39ee0374c1a7a450028dbd4ce30419378215b06c1bd2"} Jul 31 22:08:23.451427 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.451413 2554 scope.go:117] "RemoveContainer" containerID="73caba1caa76df9e9c2d39ee0374c1a7a450028dbd4ce30419378215b06c1bd2" Jul 31 22:08:23.452472 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.452440 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:08:23.452574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.452525 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:08:23.452669 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:23.452572 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:08:24.458384 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:24.458356 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" event={"ID":"b8d36c88-b636-43e4-a316-a9101dc08989","Type":"ContainerStarted","Data":"c2d24f0c4391848dd3f5b08475e276f768740f5b4411cc4db60daa8f7bd54cc3"} Jul 31 22:08:25.043131 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:25.043091 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:08:25.043350 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:25.043327 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hhkcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 31 22:08:25.043437 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:25.043385 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" podUID="b8d36c88-b636-43e4-a316-a9101dc08989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 31 22:08:26.043939 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:26.043907 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hhkcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 31 22:08:26.044309 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:26.043963 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" podUID="b8d36c88-b636-43e4-a316-a9101dc08989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 31 22:08:27.043672 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:27.043632 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hhkcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 31 22:08:27.043856 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:27.043690 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" podUID="b8d36c88-b636-43e4-a316-a9101dc08989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 31 22:08:28.043823 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:28.043789 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hhkcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 31 22:08:28.044203 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:28.043847 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" podUID="b8d36c88-b636-43e4-a316-a9101dc08989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 31 22:08:29.043963 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:29.043926 2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hhkcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 31 22:08:29.044325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:29.044011 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" podUID="b8d36c88-b636-43e4-a316-a9101dc08989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 31 22:08:30.046798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:30.046770 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:08:30.047228 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:30.047021 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:08:30.047584 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:30.047566 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hhkcc" Jul 31 22:08:35.599195 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:35.599153 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:35.668332 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:35.668296 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:42.070772 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:42.070734 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:42.449771 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:42.449739 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:08:45.581177 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:45.581132 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:45.957977 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:45.957948 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:08:53.697841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:53.697765 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:08:54.554561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:54.554523 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:08:58.638644 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:08:58.638584 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:09:05.250325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.250289 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj"] Jul 31 22:09:05.253807 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.253786 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.257331 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.257308 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 31 22:09:05.257784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.257767 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 31 22:09:05.257878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.257621 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8hckp\"" Jul 31 22:09:05.257933 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.257873 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 31 22:09:05.260813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.259709 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.260813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.259768 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.260813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.259830 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.260813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.259863 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstkk\" (UniqueName: \"kubernetes.io/projected/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-kube-api-access-nstkk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.260813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.259933 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.260813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.259997 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.268030 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.268000 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj"] Jul 31 22:09:05.358035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.357983 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:09:05.360990 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.360969 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstkk\" (UniqueName: \"kubernetes.io/projected/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-kube-api-access-nstkk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361003 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361033 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361158 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361106 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361158 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361137 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361249 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361174 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361468 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361448 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361572 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361501 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.361572 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.361540 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.363375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.363352 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.363652 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.363634 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.369290 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.369271 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstkk\" (UniqueName: \"kubernetes.io/projected/a40d0c80-1b73-4d0a-b8bd-488e3392eff0-kube-api-access-nstkk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj\" (UID: \"a40d0c80-1b73-4d0a-b8bd-488e3392eff0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.567118 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.567029 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:05.704360 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:05.704335 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj"] Jul 31 22:09:05.706256 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:09:05.706225 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40d0c80_1b73_4d0a_b8bd_488e3392eff0.slice/crio-ff6eef69291b066c2282b7691795b4477d4f11db978257c3ceb5f35ca675220c WatchSource:0}: Error finding container ff6eef69291b066c2282b7691795b4477d4f11db978257c3ceb5f35ca675220c: Status 404 returned error can't find the container with id ff6eef69291b066c2282b7691795b4477d4f11db978257c3ceb5f35ca675220c Jul 31 22:09:06.662806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:06.662750 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" event={"ID":"a40d0c80-1b73-4d0a-b8bd-488e3392eff0","Type":"ContainerStarted","Data":"ff6eef69291b066c2282b7691795b4477d4f11db978257c3ceb5f35ca675220c"} Jul 31 22:09:06.736772 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:06.736746 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:09:07.669621 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:07.669558 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" event={"ID":"a40d0c80-1b73-4d0a-b8bd-488e3392eff0","Type":"ContainerStarted","Data":"df8af1eac4f37abf79b047ac2b9f95d5485553d669405040335dce6ffd6e7a25"} Jul 31 22:09:07.692924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:07.692869 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" podStartSLOduration=1.7141828380000002 podStartE2EDuration="2.692851672s" totalImagesPullingTime="978.668834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:09:07.691826193 +0000 UTC m=+572.256674775" watchObservedRunningTime="2026-07-31 22:09:07.692851672 +0000 UTC m=+572.257700232" Jul 31 22:09:15.567475 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:15.567434 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:15.568002 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:15.567500 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:15.580060 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:15.580026 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:15.720671 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:15.720644 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj" Jul 31 22:09:16.565220 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:16.565186 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:09:18.639437 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:18.639395 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:09:23.150114 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:23.150083 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:09:25.743697 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:25.743659 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:09:35.895569 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:35.895541 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:09:35.896632 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:35.896611 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:09:53.601523 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:09:53.601485 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:10:04.835409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:10:04.835327 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:10:50.667949 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:10:50.667915 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:10:50.760723 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:10:50.760686 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:10:51.851797 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:10:51.851758 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:10:51.945792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:10:51.945751 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:11:19.316641 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:11:19.316559 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:11:29.050903 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:11:29.050821 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:11:40.855612 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:11:40.855573 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:12:36.060321 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:12:36.060288 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:12:56.970803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:12:56.970768 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:12:58.224957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:12:58.224863 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:13:21.447241 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:13:21.447204 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:13:31.650942 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:13:31.650906 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:13:41.503965 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:13:41.503929 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:13:41.605143 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:13:41.605108 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:01.621575 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:01.621538 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:01.756899 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:01.756865 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:01.864969 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:01.864934 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:07.866932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:07.866894 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:07.983713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:07.983683 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:08.073540 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:08.073504 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:08.255142 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:08.255109 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:14.454015 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:14.453975 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:14.526894 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:14.526864 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:14:29.960199 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:29.960119 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:14:35.927311 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:35.927280 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:14:35.928069 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:35.928047 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:14:37.442014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:37.441983 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:14:53.035903 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:53.035864 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:14:59.133937 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:14:59.133892 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:15:00.161905 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.161871 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg"] Jul 31 22:15:00.165690 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.165670 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" Jul 31 22:15:00.168781 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.168761 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lgfz6\"" Jul 31 22:15:00.175232 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.175216 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" Jul 31 22:15:00.195082 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.195042 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg"] Jul 31 22:15:00.317116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.317051 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg"] Jul 31 22:15:00.319503 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:15:00.319476 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fedcf1c_a50f_4b5d_a606_47892e30d78a.slice/crio-6993e7dbee0f594d758b031cf6451da5187b6c85924ef339906c8f6c7538c1f0 WatchSource:0}: Error finding container 6993e7dbee0f594d758b031cf6451da5187b6c85924ef339906c8f6c7538c1f0: Status 404 returned error can't find the container with id 6993e7dbee0f594d758b031cf6451da5187b6c85924ef339906c8f6c7538c1f0 Jul 31 22:15:00.321422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.321406 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 22:15:00.399591 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:00.399557 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" event={"ID":"4fedcf1c-a50f-4b5d-a606-47892e30d78a","Type":"ContainerStarted","Data":"6993e7dbee0f594d758b031cf6451da5187b6c85924ef339906c8f6c7538c1f0"} Jul 31 22:15:03.417941 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:03.417900 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" event={"ID":"4fedcf1c-a50f-4b5d-a606-47892e30d78a","Type":"ContainerStarted","Data":"abc7266e16a0c132de15b95d98eb9fca7f7754a41e9d99441a99eedf9f867e2e"} Jul 31 22:15:03.434906 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:03.434859 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" podStartSLOduration=1.156248533 podStartE2EDuration="3.434842947s" totalImagesPullingTime="2.278594414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:15:03.433914972 +0000 UTC m=+927.998763555" watchObservedRunningTime="2026-07-31 22:15:03.434842947 +0000 UTC m=+927.999691528" Jul 31 22:15:04.424181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:04.424145 2554 generic.go:356] "Generic (PLEG): container finished" podID="4fedcf1c-a50f-4b5d-a606-47892e30d78a" containerID="abc7266e16a0c132de15b95d98eb9fca7f7754a41e9d99441a99eedf9f867e2e" exitCode=0 Jul 31 22:15:04.424625 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:04.424232 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" event={"ID":"4fedcf1c-a50f-4b5d-a606-47892e30d78a","Type":"ContainerDied","Data":"abc7266e16a0c132de15b95d98eb9fca7f7754a41e9d99441a99eedf9f867e2e"} Jul 31 22:15:05.559920 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:05.559898 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" Jul 31 22:15:06.434869 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:06.434837 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" Jul 31 22:15:06.435021 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:06.434868 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758935-hv9gg" event={"ID":"4fedcf1c-a50f-4b5d-a606-47892e30d78a","Type":"ContainerDied","Data":"6993e7dbee0f594d758b031cf6451da5187b6c85924ef339906c8f6c7538c1f0"} Jul 31 22:15:06.435021 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:06.434918 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6993e7dbee0f594d758b031cf6451da5187b6c85924ef339906c8f6c7538c1f0" Jul 31 22:15:08.641079 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:08.641044 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:15:10.755251 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:10.755202 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:15:17.059428 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:17.059389 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:15:29.048045 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:29.048007 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:15:34.239540 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:34.239509 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:15:42.441865 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:42.441827 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:15:48.338721 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:15:48.338685 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:16:01.155455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:01.155376 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:16:12.062856 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:12.062817 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:16:19.810451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:19.810418 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:16:35.677375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:35.677340 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:16:46.243999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:46.243960 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:16:51.345222 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:51.345185 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:16:59.240358 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:16:59.240317 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:06.935822 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:06.935786 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:12.054035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:12.053998 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:17.957556 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:17.957522 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:25.645249 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:25.645209 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:39.650546 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:39.650457 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:44.953191 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:44.953152 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:17:55.441301 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:17:55.441268 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:18:00.638294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:18:00.638255 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:18:06.754727 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:18:06.754687 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:18:15.670963 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:18:15.670927 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:18:20.610789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:18:20.610752 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:18:32.552258 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:18:32.552217 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:19:27.321890 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.321850 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc"] Jul 31 22:19:27.322410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.322039 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" podUID="336cd904-3767-4dca-903c-39bd5ec13181" containerName="manager" containerID="cri-o://9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad" gracePeriod=10 Jul 31 22:19:27.583437 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.583380 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:19:27.690035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.690004 2554 generic.go:356] "Generic (PLEG): container finished" podID="336cd904-3767-4dca-903c-39bd5ec13181" containerID="9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad" exitCode=0 Jul 31 22:19:27.690181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.690049 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" Jul 31 22:19:27.690181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.690053 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" event={"ID":"336cd904-3767-4dca-903c-39bd5ec13181","Type":"ContainerDied","Data":"9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad"} Jul 31 22:19:27.690263 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.690179 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc" event={"ID":"336cd904-3767-4dca-903c-39bd5ec13181","Type":"ContainerDied","Data":"f7d68f2b9c60509a14e60a47b8b87979b69ce8fd64745715becb841dd11b1638"} Jul 31 22:19:27.690263 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.690207 2554 scope.go:117] "RemoveContainer" containerID="9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad" Jul 31 22:19:27.699325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.699306 2554 scope.go:117] "RemoveContainer" containerID="9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad" Jul 31 22:19:27.699567 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:19:27.699545 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad\": container with ID starting with 9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad not found: ID does not exist" containerID="9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad" Jul 31 22:19:27.699649 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.699580 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad"} err="failed to get container status \"9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad\": rpc error: code = NotFound desc = could not find container \"9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad\": container with ID starting with 9453c84fb12ee557d6283b802bedc14bc8ad240b3efd1f1af0d3dc45bf1be2ad not found: ID does not exist" Jul 31 22:19:27.721947 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.721927 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/336cd904-3767-4dca-903c-39bd5ec13181-extensions-socket-volume\") pod \"336cd904-3767-4dca-903c-39bd5ec13181\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " Jul 31 22:19:27.722058 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.721986 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfpfp\" (UniqueName: \"kubernetes.io/projected/336cd904-3767-4dca-903c-39bd5ec13181-kube-api-access-tfpfp\") pod \"336cd904-3767-4dca-903c-39bd5ec13181\" (UID: \"336cd904-3767-4dca-903c-39bd5ec13181\") " Jul 31 22:19:27.722276 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.722254 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336cd904-3767-4dca-903c-39bd5ec13181-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "336cd904-3767-4dca-903c-39bd5ec13181" (UID: "336cd904-3767-4dca-903c-39bd5ec13181"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:19:27.723889 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.723867 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336cd904-3767-4dca-903c-39bd5ec13181-kube-api-access-tfpfp" (OuterVolumeSpecName: "kube-api-access-tfpfp") pod "336cd904-3767-4dca-903c-39bd5ec13181" (UID: "336cd904-3767-4dca-903c-39bd5ec13181"). InnerVolumeSpecName "kube-api-access-tfpfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:19:27.823423 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.823388 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfpfp\" (UniqueName: \"kubernetes.io/projected/336cd904-3767-4dca-903c-39bd5ec13181-kube-api-access-tfpfp\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:19:27.823423 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:27.823417 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/336cd904-3767-4dca-903c-39bd5ec13181-extensions-socket-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:19:28.008026 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:28.007998 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc"] Jul 31 22:19:28.014043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:28.014017 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-flgjc"] Jul 31 22:19:29.957446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:29.957409 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="336cd904-3767-4dca-903c-39bd5ec13181" path="/var/lib/kubelet/pods/336cd904-3767-4dca-903c-39bd5ec13181/volumes" Jul 31 22:19:35.956045 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:35.956013 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:19:35.958365 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:19:35.958335 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:20:28.074417 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074381 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h"] Jul 31 22:20:28.074923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074827 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fedcf1c-a50f-4b5d-a606-47892e30d78a" containerName="cleanup" Jul 31 22:20:28.074923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074842 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fedcf1c-a50f-4b5d-a606-47892e30d78a" containerName="cleanup" Jul 31 22:20:28.074923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074857 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="336cd904-3767-4dca-903c-39bd5ec13181" containerName="manager" Jul 31 22:20:28.074923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074864 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="336cd904-3767-4dca-903c-39bd5ec13181" containerName="manager" Jul 31 22:20:28.075121 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074948 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fedcf1c-a50f-4b5d-a606-47892e30d78a" containerName="cleanup" Jul 31 22:20:28.075121 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.074962 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="336cd904-3767-4dca-903c-39bd5ec13181" containerName="manager" Jul 31 22:20:28.078073 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.078057 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.082087 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.082067 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2sxtx\"" Jul 31 22:20:28.097321 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.097297 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h"] Jul 31 22:20:28.228761 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.228671 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnkqr\" (UniqueName: \"kubernetes.io/projected/2c12ec64-59eb-478d-9c86-df289b4004a8-kube-api-access-fnkqr\") pod \"kuadrant-operator-controller-manager-68cf589589-gq89h\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.228761 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.228723 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c12ec64-59eb-478d-9c86-df289b4004a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-gq89h\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.329845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.329758 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnkqr\" (UniqueName: \"kubernetes.io/projected/2c12ec64-59eb-478d-9c86-df289b4004a8-kube-api-access-fnkqr\") pod \"kuadrant-operator-controller-manager-68cf589589-gq89h\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.329845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.329812 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c12ec64-59eb-478d-9c86-df289b4004a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-gq89h\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.330234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.330214 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c12ec64-59eb-478d-9c86-df289b4004a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-gq89h\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.338656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.338620 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnkqr\" (UniqueName: \"kubernetes.io/projected/2c12ec64-59eb-478d-9c86-df289b4004a8-kube-api-access-fnkqr\") pod \"kuadrant-operator-controller-manager-68cf589589-gq89h\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.388236 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.388211 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:28.526951 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.526863 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h"] Jul 31 22:20:28.529350 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:20:28.529326 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c12ec64_59eb_478d_9c86_df289b4004a8.slice/crio-c385ccfb516c65bcf48ca38073005315e3366e8a86629324b1891f518982737b WatchSource:0}: Error finding container c385ccfb516c65bcf48ca38073005315e3366e8a86629324b1891f518982737b: Status 404 returned error can't find the container with id c385ccfb516c65bcf48ca38073005315e3366e8a86629324b1891f518982737b Jul 31 22:20:28.980750 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.980641 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" event={"ID":"2c12ec64-59eb-478d-9c86-df289b4004a8","Type":"ContainerStarted","Data":"618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292"} Jul 31 22:20:28.980750 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.980710 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" event={"ID":"2c12ec64-59eb-478d-9c86-df289b4004a8","Type":"ContainerStarted","Data":"c385ccfb516c65bcf48ca38073005315e3366e8a86629324b1891f518982737b"} Jul 31 22:20:28.981146 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:28.980853 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:29.008918 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:29.008865 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" podStartSLOduration=1.008848347 podStartE2EDuration="1.008848347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:20:29.006658188 +0000 UTC m=+1253.571506766" watchObservedRunningTime="2026-07-31 22:20:29.008848347 +0000 UTC m=+1253.573696929" Jul 31 22:20:39.988519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:39.988484 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:20:46.944954 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:46.944917 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:20:52.164368 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:20:52.164333 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:21:16.484375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:16.484338 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:21:19.650044 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:19.649997 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:21:20.808534 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:20.808495 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:21:22.599857 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:22.599822 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:21:31.349452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:31.349415 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:21:37.923178 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:37.923141 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:21:47.263882 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:47.263846 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:21:52.746592 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:21:52.746552 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:03.652540 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:03.652458 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:14.568773 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:14.568737 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:19.050277 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:19.050240 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:24.152872 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:24.152831 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:33.451147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:33.451109 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:40.260891 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:40.260847 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:22:49.453687 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:22:49.453643 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:23:11.647095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:11.647062 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:23:18.581072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:18.581023 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:23:30.174474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.174392 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr"] Jul 31 22:23:30.178487 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.178468 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.181375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.181354 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-84d6c68e-kserve-self-signed-certs\"" Jul 31 22:23:30.197886 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.197853 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tmp-dir\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.197996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.197891 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t65v\" (UniqueName: \"kubernetes.io/projected/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-kube-api-access-2t65v\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.197996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.197921 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-dshm\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.198092 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.198047 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-home\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.198092 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.198086 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tls-certs\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.198211 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.198190 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-model-cache\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.200134 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.200110 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr"] Jul 31 22:23:30.266986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.266953 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:23:30.299288 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299261 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tls-certs\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299414 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299323 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-model-cache\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299414 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299343 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tmp-dir\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299414 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299359 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t65v\" (UniqueName: \"kubernetes.io/projected/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-kube-api-access-2t65v\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299414 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299378 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-dshm\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299625 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299493 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-home\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299928 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299899 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tmp-dir\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.299928 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299915 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-model-cache\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.300092 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.299957 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-home\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.301739 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.301711 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-dshm\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.301918 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.301901 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tls-certs\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.309372 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.309342 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t65v\" (UniqueName: \"kubernetes.io/projected/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-kube-api-access-2t65v\") pod \"e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.488440 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.488397 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:30.630965 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.630940 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr"] Jul 31 22:23:30.633433 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:23:30.633407 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ba98a80_2e46_4448_a2d0_cb04dd4eb0e0.slice/crio-5d1e911ae24c6cd596e2be7a3742fdc5095eedec5e4f712d63130381395737e4 WatchSource:0}: Error finding container 5d1e911ae24c6cd596e2be7a3742fdc5095eedec5e4f712d63130381395737e4: Status 404 returned error can't find the container with id 5d1e911ae24c6cd596e2be7a3742fdc5095eedec5e4f712d63130381395737e4 Jul 31 22:23:30.891514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.891417 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" event={"ID":"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0","Type":"ContainerStarted","Data":"81d1e6235e0803b8afe9dc1e5e71518744931f1f7f840fcf15e9461e7a48a5f6"} Jul 31 22:23:30.891514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.891461 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" event={"ID":"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0","Type":"ContainerStarted","Data":"5d1e911ae24c6cd596e2be7a3742fdc5095eedec5e4f712d63130381395737e4"} Jul 31 22:23:30.909569 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:30.909520 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" podStartSLOduration=0.909505923 podStartE2EDuration="909.505923ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:23:30.908438832 +0000 UTC m=+1435.473287415" watchObservedRunningTime="2026-07-31 22:23:30.909505923 +0000 UTC m=+1435.474354504" Jul 31 22:23:33.655474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:33.655440 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:23:40.489190 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:40.489142 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:40.489582 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:40.489209 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:40.503466 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:40.503435 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:40.955427 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:40.955394 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:23:50.345129 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.345095 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t"] Jul 31 22:23:50.348961 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.348940 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.351668 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.351646 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-84d6c68e-kserve-self-signed-certs\"" Jul 31 22:23:50.359300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.359277 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t"] Jul 31 22:23:50.387666 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.387636 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:23:50.464932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.464893 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpvmt\" (UniqueName: \"kubernetes.io/projected/3b53f7ff-8953-40ca-975d-b9d39aac6500-kube-api-access-tpvmt\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.465112 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.464993 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-model-cache\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.465112 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.465039 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-dshm\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.465112 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.465063 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-home\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.465112 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.465085 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-tmp-dir\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.465276 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.465157 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b53f7ff-8953-40ca-975d-b9d39aac6500-tls-certs\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566411 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-tmp-dir\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566448 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b53f7ff-8953-40ca-975d-b9d39aac6500-tls-certs\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566744 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566482 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpvmt\" (UniqueName: \"kubernetes.io/projected/3b53f7ff-8953-40ca-975d-b9d39aac6500-kube-api-access-tpvmt\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566744 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566529 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-model-cache\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566744 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566555 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-dshm\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566744 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566656 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-home\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.566966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566911 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-tmp-dir\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.567007 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.566984 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-model-cache\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.567065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.567046 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-home\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.568818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.568791 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-dshm\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.569212 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.569189 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b53f7ff-8953-40ca-975d-b9d39aac6500-tls-certs\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.575848 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.575822 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpvmt\" (UniqueName: \"kubernetes.io/projected/3b53f7ff-8953-40ca-975d-b9d39aac6500-kube-api-access-tpvmt\") pod \"e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.658949 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.658856 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:23:50.697063 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.697034 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:23:50.796295 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.796272 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t"] Jul 31 22:23:50.798541 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:23:50.798500 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b53f7ff_8953_40ca_975d_b9d39aac6500.slice/crio-0a8f454bf4c5f348f8ea7f7efe786704e550a11787422b2046a4e67629fabd7f WatchSource:0}: Error finding container 0a8f454bf4c5f348f8ea7f7efe786704e550a11787422b2046a4e67629fabd7f: Status 404 returned error can't find the container with id 0a8f454bf4c5f348f8ea7f7efe786704e550a11787422b2046a4e67629fabd7f Jul 31 22:23:50.994877 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.994839 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" event={"ID":"3b53f7ff-8953-40ca-975d-b9d39aac6500","Type":"ContainerStarted","Data":"58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a"} Jul 31 22:23:50.995043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:50.994887 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" event={"ID":"3b53f7ff-8953-40ca-975d-b9d39aac6500","Type":"ContainerStarted","Data":"0a8f454bf4c5f348f8ea7f7efe786704e550a11787422b2046a4e67629fabd7f"} Jul 31 22:23:51.014139 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:51.014093 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" podStartSLOduration=1.014078626 podStartE2EDuration="1.014078626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:23:51.012661176 +0000 UTC m=+1455.577509760" watchObservedRunningTime="2026-07-31 22:23:51.014078626 +0000 UTC m=+1455.578927248" Jul 31 22:23:52.947646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:23:52.947614 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:00.659203 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:00.659133 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:24:00.659682 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:00.659220 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:24:00.671551 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:00.671528 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:24:01.053879 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:01.053845 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:24:07.899569 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:07.899534 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr"] Jul 31 22:24:07.917389 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:07.917360 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:24:08.697711 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:08.697674 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t"] Jul 31 22:24:08.729154 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:08.729115 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:24:10.653261 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:10.653225 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:20.369357 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:20.369322 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:22.919296 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:22.919257 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" podUID="2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" containerName="main" containerID="cri-o://81d1e6235e0803b8afe9dc1e5e71518744931f1f7f840fcf15e9461e7a48a5f6" gracePeriod=45 Jul 31 22:24:23.157444 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.157414 2554 generic.go:356] "Generic (PLEG): container finished" podID="2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" containerID="81d1e6235e0803b8afe9dc1e5e71518744931f1f7f840fcf15e9461e7a48a5f6" exitCode=0 Jul 31 22:24:23.157579 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.157487 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" event={"ID":"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0","Type":"ContainerDied","Data":"81d1e6235e0803b8afe9dc1e5e71518744931f1f7f840fcf15e9461e7a48a5f6"} Jul 31 22:24:23.178076 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.178017 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:24:23.272683 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272650 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-model-cache\") pod \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " Jul 31 22:24:23.272849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272693 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-dshm\") pod \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " Jul 31 22:24:23.272849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272741 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t65v\" (UniqueName: \"kubernetes.io/projected/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-kube-api-access-2t65v\") pod \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " Jul 31 22:24:23.272849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272770 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tmp-dir\") pod \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " Jul 31 22:24:23.272849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272808 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-home\") pod \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " Jul 31 22:24:23.272849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272835 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tls-certs\") pod \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\" (UID: \"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0\") " Jul 31 22:24:23.273115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.272926 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" (UID: "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:23.273115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.273075 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-home" (OuterVolumeSpecName: "home") pod "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" (UID: "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:23.273115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.273090 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" (UID: "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:23.273267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.273200 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tmp-dir\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:23.273267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.273223 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-home\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:23.273267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.273237 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-model-cache\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:23.274957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.274930 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-dshm" (OuterVolumeSpecName: "dshm") pod "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" (UID: "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:23.275077 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.274994 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" (UID: "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:24:23.275077 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.274999 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-kube-api-access-2t65v" (OuterVolumeSpecName: "kube-api-access-2t65v") pod "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" (UID: "2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0"). InnerVolumeSpecName "kube-api-access-2t65v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:24:23.373752 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.373712 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-tls-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:23.373752 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.373742 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-dshm\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:23.373752 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.373754 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t65v\" (UniqueName: \"kubernetes.io/projected/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0-kube-api-access-2t65v\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:23.716757 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.716713 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" podUID="3b53f7ff-8953-40ca-975d-b9d39aac6500" containerName="main" containerID="cri-o://58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a" gracePeriod=45 Jul 31 22:24:23.964070 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:23.964043 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:24:24.078802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.078713 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-tmp-dir\") pod \"3b53f7ff-8953-40ca-975d-b9d39aac6500\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " Jul 31 22:24:24.078802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.078758 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-dshm\") pod \"3b53f7ff-8953-40ca-975d-b9d39aac6500\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " Jul 31 22:24:24.078802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.078796 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b53f7ff-8953-40ca-975d-b9d39aac6500-tls-certs\") pod \"3b53f7ff-8953-40ca-975d-b9d39aac6500\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " Jul 31 22:24:24.079080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.078828 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpvmt\" (UniqueName: \"kubernetes.io/projected/3b53f7ff-8953-40ca-975d-b9d39aac6500-kube-api-access-tpvmt\") pod \"3b53f7ff-8953-40ca-975d-b9d39aac6500\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " Jul 31 22:24:24.079080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.078852 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-model-cache\") pod \"3b53f7ff-8953-40ca-975d-b9d39aac6500\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " Jul 31 22:24:24.079080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.078907 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-home\") pod \"3b53f7ff-8953-40ca-975d-b9d39aac6500\" (UID: \"3b53f7ff-8953-40ca-975d-b9d39aac6500\") " Jul 31 22:24:24.079080 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.079009 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b53f7ff-8953-40ca-975d-b9d39aac6500" (UID: "3b53f7ff-8953-40ca-975d-b9d39aac6500"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:24.079289 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.079172 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b53f7ff-8953-40ca-975d-b9d39aac6500" (UID: "3b53f7ff-8953-40ca-975d-b9d39aac6500"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:24.079346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.079293 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-home" (OuterVolumeSpecName: "home") pod "3b53f7ff-8953-40ca-975d-b9d39aac6500" (UID: "3b53f7ff-8953-40ca-975d-b9d39aac6500"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:24.079346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.079301 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-tmp-dir\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:24.079346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.079321 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-model-cache\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:24.081068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.081045 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b53f7ff-8953-40ca-975d-b9d39aac6500-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b53f7ff-8953-40ca-975d-b9d39aac6500" (UID: "3b53f7ff-8953-40ca-975d-b9d39aac6500"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:24:24.081345 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.081329 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b53f7ff-8953-40ca-975d-b9d39aac6500-kube-api-access-tpvmt" (OuterVolumeSpecName: "kube-api-access-tpvmt") pod "3b53f7ff-8953-40ca-975d-b9d39aac6500" (UID: "3b53f7ff-8953-40ca-975d-b9d39aac6500"). InnerVolumeSpecName "kube-api-access-tpvmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:24:24.081406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.081337 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-dshm" (OuterVolumeSpecName: "dshm") pod "3b53f7ff-8953-40ca-975d-b9d39aac6500" (UID: "3b53f7ff-8953-40ca-975d-b9d39aac6500"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:24:24.164204 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.164179 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" Jul 31 22:24:24.164204 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.164194 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr" event={"ID":"2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0","Type":"ContainerDied","Data":"5d1e911ae24c6cd596e2be7a3742fdc5095eedec5e4f712d63130381395737e4"} Jul 31 22:24:24.164445 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.164244 2554 scope.go:117] "RemoveContainer" containerID="81d1e6235e0803b8afe9dc1e5e71518744931f1f7f840fcf15e9461e7a48a5f6" Jul 31 22:24:24.165502 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.165482 2554 generic.go:356] "Generic (PLEG): container finished" podID="3b53f7ff-8953-40ca-975d-b9d39aac6500" containerID="58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a" exitCode=0 Jul 31 22:24:24.165624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.165526 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" Jul 31 22:24:24.165624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.165573 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" event={"ID":"3b53f7ff-8953-40ca-975d-b9d39aac6500","Type":"ContainerDied","Data":"58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a"} Jul 31 22:24:24.165725 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.165628 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t" event={"ID":"3b53f7ff-8953-40ca-975d-b9d39aac6500","Type":"ContainerDied","Data":"0a8f454bf4c5f348f8ea7f7efe786704e550a11787422b2046a4e67629fabd7f"} Jul 31 22:24:24.172976 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.172959 2554 scope.go:117] "RemoveContainer" containerID="58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a" Jul 31 22:24:24.180319 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.180301 2554 scope.go:117] "RemoveContainer" containerID="58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a" Jul 31 22:24:24.180395 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.180358 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-home\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:24.180395 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.180378 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b53f7ff-8953-40ca-975d-b9d39aac6500-dshm\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:24.180395 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.180390 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b53f7ff-8953-40ca-975d-b9d39aac6500-tls-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:24.180546 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.180404 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpvmt\" (UniqueName: \"kubernetes.io/projected/3b53f7ff-8953-40ca-975d-b9d39aac6500-kube-api-access-tpvmt\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:24:24.180634 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:24:24.180580 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a\": container with ID starting with 58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a not found: ID does not exist" containerID="58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a" Jul 31 22:24:24.180634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.180618 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a"} err="failed to get container status \"58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a\": rpc error: code = NotFound desc = could not find container \"58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a\": container with ID starting with 58ff354663b1043621b510dddc78c7b77e431871e0397acbe0b9f898ca38ce2a not found: ID does not exist" Jul 31 22:24:24.186671 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.186652 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr"] Jul 31 22:24:24.191830 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.191810 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-84d6c68e-kserve-f4f6686dc-l7ggr"] Jul 31 22:24:24.232181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.232154 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t"] Jul 31 22:24:24.235160 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:24.235139 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-84d6c68e-kserve-76fbc9dc95-xk22t"] Jul 31 22:24:25.957258 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:25.957215 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" path="/var/lib/kubelet/pods/2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0/volumes" Jul 31 22:24:25.957797 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:25.957777 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b53f7ff-8953-40ca-975d-b9d39aac6500" path="/var/lib/kubelet/pods/3b53f7ff-8953-40ca-975d-b9d39aac6500/volumes" Jul 31 22:24:27.090469 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:27.090426 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:35.985860 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:35.985830 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:24:35.988832 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:35.988809 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:24:36.247689 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:36.247593 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:43.452980 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:43.452941 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:50.254356 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:50.254321 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:24:57.049809 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:24:57.049775 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:25:05.651696 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:25:05.651589 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:25:15.949624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:25:15.949566 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:25:22.945681 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:25:22.945645 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:25:38.458214 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:25:38.458175 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:25:45.552983 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:25:45.552944 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:25:57.359693 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:25:57.359654 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:04.349166 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:04.349127 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:10.745362 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:10.745323 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:16.444760 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:16.444718 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:22.942979 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:22.942942 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:24.771647 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:24.771588 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:26:26.206796 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:26.206753 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:26:32.453879 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:32.453665 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:38.940083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:38.940047 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:49.344124 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:49.344080 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:26:55.638019 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:26:55.637980 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:06.642063 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:06.642020 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:21.843118 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:21.843080 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:27.742724 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:27.742682 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:31.556477 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:31.556441 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:27:34.145703 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:34.145672 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:35.114683 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:35.114641 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:27:35.214983 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:35.214951 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:27:45.930749 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:45.930712 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:54.236115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:54.236077 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:27:57.515646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:57.515576 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:27:59.448544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:27:59.448449 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:28:00.777221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:00.777181 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:00.877623 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:00.877556 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:01.258066 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:01.258029 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:06.989264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:06.989226 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:07.049269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.049228 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:07.226061 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.226024 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:07.534352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534320 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4"] Jul 31 22:28:07.534793 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534775 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" containerName="main" Jul 31 22:28:07.534793 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534793 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" containerName="main" Jul 31 22:28:07.534929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534803 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b53f7ff-8953-40ca-975d-b9d39aac6500" containerName="main" Jul 31 22:28:07.534929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534809 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b53f7ff-8953-40ca-975d-b9d39aac6500" containerName="main" Jul 31 22:28:07.534929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534885 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba98a80-2e46-4448-a2d0-cb04dd4eb0e0" containerName="main" Jul 31 22:28:07.534929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.534895 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b53f7ff-8953-40ca-975d-b9d39aac6500" containerName="main" Jul 31 22:28:07.537821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.537806 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.540665 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.540642 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-48fa749f-openshift-default-dockercfg-ngz62\"" Jul 31 22:28:07.564784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.564760 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4"] Jul 31 22:28:07.568956 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.568896 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:07.583355 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583325 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-data\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583360 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-envoy\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583381 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt6k7\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-kube-api-access-lt6k7\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583436 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-certs\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583464 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d12de9-97be-4410-bc31-37682fa2fe9e-istiod-ca-cert\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583634 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-credential-socket\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583680 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-socket\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583812 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583717 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-podinfo\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.583812 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.583797 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-token\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.684798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.684760 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-token\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.684997 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.684809 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-data\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.684997 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.684841 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-envoy\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.684997 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.684867 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt6k7\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-kube-api-access-lt6k7\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.684997 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.684894 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-certs\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685114 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d12de9-97be-4410-bc31-37682fa2fe9e-istiod-ca-cert\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685537 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685430 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-certs\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685537 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685464 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-credential-socket\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685537 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685488 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-data\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685537 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685508 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-socket\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685851 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685554 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-podinfo\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685851 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685704 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-credential-socket\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685963 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685905 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d12de9-97be-4410-bc31-37682fa2fe9e-istiod-ca-cert\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.685963 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.685952 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-socket\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.687729 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.687705 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-envoy\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.688060 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.688039 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-podinfo\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.693278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.693247 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-token\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.693392 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.693353 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt6k7\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-kube-api-access-lt6k7\") pod \"e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.809354 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.809258 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:07.848510 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.848473 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:07.996280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:07.996254 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4"] Jul 31 22:28:07.998207 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:28:07.998171 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d12de9_97be_4410_bc31_37682fa2fe9e.slice/crio-4766ee8ab32b611dd52a387219d2f5f7913f20c06bb196ec8faa27a92d7f0bd9 WatchSource:0}: Error finding container 4766ee8ab32b611dd52a387219d2f5f7913f20c06bb196ec8faa27a92d7f0bd9: Status 404 returned error can't find the container with id 4766ee8ab32b611dd52a387219d2f5f7913f20c06bb196ec8faa27a92d7f0bd9 Jul 31 22:28:08.000385 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.000356 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:08.000474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.000420 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:08.000474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.000447 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:08.283395 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.283361 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" event={"ID":"56d12de9-97be-4410-bc31-37682fa2fe9e","Type":"ContainerStarted","Data":"03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306"} Jul 31 22:28:08.283583 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.283421 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" event={"ID":"56d12de9-97be-4410-bc31-37682fa2fe9e","Type":"ContainerStarted","Data":"4766ee8ab32b611dd52a387219d2f5f7913f20c06bb196ec8faa27a92d7f0bd9"} Jul 31 22:28:08.304274 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.304220 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podStartSLOduration=1.304203802 podStartE2EDuration="1.304203802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:28:08.302785118 +0000 UTC m=+1712.867633701" watchObservedRunningTime="2026-07-31 22:28:08.304203802 +0000 UTC m=+1712.869052383" Jul 31 22:28:08.849272 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:08.849231 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:09.653038 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:09.653008 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:09.744231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:09.744195 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:09.855710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:09.855681 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:10.293253 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:10.293217 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:10.294151 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:10.294130 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:10.776141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:10.776108 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:11.288467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:11.288435 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:11.388099 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:11.388062 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:14.333566 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:14.333533 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:28:24.367513 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:24.367476 2554 generic.go:356] "Generic (PLEG): container finished" podID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerID="03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306" exitCode=0 Jul 31 22:28:24.367996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:24.367529 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" event={"ID":"56d12de9-97be-4410-bc31-37682fa2fe9e","Type":"ContainerDied","Data":"03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306"} Jul 31 22:28:24.367996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:24.367949 2554 scope.go:117] "RemoveContainer" containerID="03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306" Jul 31 22:28:24.369011 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:24.368979 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:24.369119 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:24.369079 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:24.369187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:24.369128 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:25.293661 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:25.293610 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:25.375163 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:25.375126 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" event={"ID":"56d12de9-97be-4410-bc31-37682fa2fe9e","Type":"ContainerStarted","Data":"5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b"} Jul 31 22:28:25.849052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:25.849014 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:25.850314 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:25.850287 2554 patch_prober.go:28] interesting pod/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 31 22:28:25.850435 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:25.850341 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 31 22:28:26.849388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:26.849352 2554 patch_prober.go:28] interesting pod/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 31 22:28:26.849784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:26.849409 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 31 22:28:27.849312 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:27.849275 2554 patch_prober.go:28] interesting pod/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 31 22:28:27.849482 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:27.849338 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 31 22:28:28.849264 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:28.849229 2554 patch_prober.go:28] interesting pod/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 31 22:28:28.849426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:28.849295 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 31 22:28:29.849355 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:29.849310 2554 patch_prober.go:28] interesting pod/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 31 22:28:29.849862 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:29.849388 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 31 22:28:30.852671 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:30.852638 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:30.853066 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:30.852912 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:30.853678 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:30.853660 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:51.438078 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.438045 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:51.508838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.508806 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:51.573999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.573963 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:51.631843 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.631805 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:51.763576 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.763482 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4"] Jul 31 22:28:51.765197 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.765163 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" containerID="cri-o://5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b" gracePeriod=30 Jul 31 22:28:51.787016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:51.786943 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:52.033653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.033521 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:52.298785 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.298651 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:52.395243 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.395193 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:52.569988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.569889 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:52.827253 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.827175 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x"] Jul 31 22:28:52.830746 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.830726 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.833317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.833301 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-048b8d6f-gw-openshift-default-dockercfg-sb8c7\"" Jul 31 22:28:52.843642 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.843613 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x"] Jul 31 22:28:52.863815 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.863789 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:52.905163 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905125 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-certs\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905318 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905170 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-envoy\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905318 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905189 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-socket\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905318 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-credential-socket\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905318 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905239 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef86f03-9f41-4934-8e40-dd62631ed258-istiod-ca-cert\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905318 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905301 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-token\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905332 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqdmj\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-kube-api-access-mqdmj\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905366 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-podinfo\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:52.905525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:52.905409 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-data\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.006653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006620 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-data\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.006829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006723 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-certs\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.006829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006761 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-envoy\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.006829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006784 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-socket\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.006829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006808 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-credential-socket\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006832 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef86f03-9f41-4934-8e40-dd62631ed258-istiod-ca-cert\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-token\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.006996 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqdmj\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-kube-api-access-mqdmj\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.007050 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-podinfo\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.007195 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-data\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007485 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.007459 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-certs\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007637 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.007532 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-socket\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007637 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.007578 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-credential-socket\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.007912 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.007889 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef86f03-9f41-4934-8e40-dd62631ed258-istiod-ca-cert\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.009181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.009158 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-envoy\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.009761 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.009741 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-podinfo\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.014933 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.014905 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-token\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.015445 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.015424 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqdmj\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-kube-api-access-mqdmj\") pod \"e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.142292 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.142205 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:53.214403 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.214371 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:53.283994 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.283501 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x"] Jul 31 22:28:53.290924 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:28:53.290878 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef86f03_9f41_4934_8e40_dd62631ed258.slice/crio-40e878eb34b30c7ccbcbebbad421226667a2617ac89c20dc20514a643628f822 WatchSource:0}: Error finding container 40e878eb34b30c7ccbcbebbad421226667a2617ac89c20dc20514a643628f822: Status 404 returned error can't find the container with id 40e878eb34b30c7ccbcbebbad421226667a2617ac89c20dc20514a643628f822 Jul 31 22:28:53.293406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.293363 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:53.293512 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.293451 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:53.293512 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.293489 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:28:53.533897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.533855 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" event={"ID":"2ef86f03-9f41-4934-8e40-dd62631ed258","Type":"ContainerStarted","Data":"a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762"} Jul 31 22:28:53.534098 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.533919 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" event={"ID":"2ef86f03-9f41-4934-8e40-dd62631ed258","Type":"ContainerStarted","Data":"40e878eb34b30c7ccbcbebbad421226667a2617ac89c20dc20514a643628f822"} Jul 31 22:28:53.556425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.556366 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podStartSLOduration=1.556344985 podStartE2EDuration="1.556344985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:28:53.5529529 +0000 UTC m=+1758.117801497" watchObservedRunningTime="2026-07-31 22:28:53.556344985 +0000 UTC m=+1758.121193569" Jul 31 22:28:53.735912 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:53.735867 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:28:54.142411 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:54.142380 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:54.147019 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:54.146999 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:54.539014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:54.538986 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:54.540062 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:54.540038 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:28:55.022326 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:55.022291 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:55.133428 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:55.133392 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:55.628409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:55.628370 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:55.729100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:55.729066 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:55.817869 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:55.817827 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:28:57.034929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.034870 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:57.145829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.145797 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-credential-socket\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.145829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.145829 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-token\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.145868 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-certs\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.145887 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-envoy\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.145926 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d12de9-97be-4410-bc31-37682fa2fe9e-istiod-ca-cert\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.145977 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt6k7\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-kube-api-access-lt6k7\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146041 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-data\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146068 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-socket\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146387 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146091 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-podinfo\") pod \"56d12de9-97be-4410-bc31-37682fa2fe9e\" (UID: \"56d12de9-97be-4410-bc31-37682fa2fe9e\") " Jul 31 22:28:57.146387 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146122 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:28:57.146387 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146210 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:28:57.146527 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146394 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.146527 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146414 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.146527 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146420 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d12de9-97be-4410-bc31-37682fa2fe9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:28:57.146527 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146455 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:28:57.146714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.146470 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:28:57.148346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.148325 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:28:57.148489 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.148471 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-kube-api-access-lt6k7" (OuterVolumeSpecName: "kube-api-access-lt6k7") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "kube-api-access-lt6k7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:28:57.148557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.148511 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:28:57.148557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.148518 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56d12de9-97be-4410-bc31-37682fa2fe9e" (UID: "56d12de9-97be-4410-bc31-37682fa2fe9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:28:57.247028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.246992 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d12de9-97be-4410-bc31-37682fa2fe9e-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.247028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.247022 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt6k7\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-kube-api-access-lt6k7\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.247028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.247035 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.247280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.247049 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.247280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.247060 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.247280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.247071 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.247280 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.247081 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d12de9-97be-4410-bc31-37682fa2fe9e-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:28:57.555242 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.555209 2554 generic.go:356] "Generic (PLEG): container finished" podID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerID="5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b" exitCode=0 Jul 31 22:28:57.555422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.555250 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" event={"ID":"56d12de9-97be-4410-bc31-37682fa2fe9e","Type":"ContainerDied","Data":"5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b"} Jul 31 22:28:57.555422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.555259 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" Jul 31 22:28:57.555422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.555293 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4" event={"ID":"56d12de9-97be-4410-bc31-37682fa2fe9e","Type":"ContainerDied","Data":"4766ee8ab32b611dd52a387219d2f5f7913f20c06bb196ec8faa27a92d7f0bd9"} Jul 31 22:28:57.555422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.555316 2554 scope.go:117] "RemoveContainer" containerID="5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b" Jul 31 22:28:57.564690 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.564667 2554 scope.go:117] "RemoveContainer" containerID="03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306" Jul 31 22:28:57.572606 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.572575 2554 scope.go:117] "RemoveContainer" containerID="5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b" Jul 31 22:28:57.572857 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:28:57.572838 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b\": container with ID starting with 5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b not found: ID does not exist" containerID="5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b" Jul 31 22:28:57.572920 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.572864 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b"} err="failed to get container status \"5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b\": rpc error: code = NotFound desc = could not find container \"5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b\": container with ID starting with 5388feb5ff8d91c8e3d7db09f1603cdef229887f1c4bd4f5a8afe7c57c19d59b not found: ID does not exist" Jul 31 22:28:57.572920 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.572881 2554 scope.go:117] "RemoveContainer" containerID="03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306" Jul 31 22:28:57.573081 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:28:57.573063 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306\": container with ID starting with 03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306 not found: ID does not exist" containerID="03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306" Jul 31 22:28:57.573121 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.573087 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306"} err="failed to get container status \"03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306\": rpc error: code = NotFound desc = could not find container \"03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306\": container with ID starting with 03661c233816d07f5daa1fb949fb7941b30b81665f9ee308f128dcef94766306 not found: ID does not exist" Jul 31 22:28:57.593552 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.593518 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4"] Jul 31 22:28:57.595646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.595622 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-48fa749f-openshift-default-5b9f888cbb-x2ww4"] Jul 31 22:28:57.957523 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:28:57.957489 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" path="/var/lib/kubelet/pods/56d12de9-97be-4410-bc31-37682fa2fe9e/volumes" Jul 31 22:29:03.578346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:03.578310 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:29:08.630670 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:08.630640 2554 generic.go:356] "Generic (PLEG): container finished" podID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerID="a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762" exitCode=0 Jul 31 22:29:08.631122 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:08.630713 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" event={"ID":"2ef86f03-9f41-4934-8e40-dd62631ed258","Type":"ContainerDied","Data":"a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762"} Jul 31 22:29:08.631173 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:08.631139 2554 scope.go:117] "RemoveContainer" containerID="a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762" Jul 31 22:29:08.632182 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:08.632151 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:08.632297 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:08.632236 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:08.632297 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:08.632286 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:09.539448 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:09.539411 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:09.638259 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:09.638224 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" event={"ID":"2ef86f03-9f41-4934-8e40-dd62631ed258","Type":"ContainerStarted","Data":"f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c"} Jul 31 22:29:10.142806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:10.142766 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:10.143841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:10.143818 2554 patch_prober.go:28] interesting pod/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 31 22:29:10.143947 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:10.143876 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 31 22:29:11.142853 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:11.142816 2554 patch_prober.go:28] interesting pod/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 31 22:29:11.143265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:11.142872 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 31 22:29:12.143363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:12.143328 2554 patch_prober.go:28] interesting pod/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 31 22:29:12.143792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:12.143388 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 31 22:29:13.143139 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:13.143104 2554 patch_prober.go:28] interesting pod/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 31 22:29:13.143316 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:13.143189 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 31 22:29:14.143094 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:14.143058 2554 patch_prober.go:28] interesting pod/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 31 22:29:14.143465 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:14.143120 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 31 22:29:15.147346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:15.147313 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:15.147803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:15.147637 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:15.148412 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:15.148393 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:26.551380 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.551336 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:26.648699 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.648652 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:26.729863 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.729821 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:26.798438 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.798403 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:26.915883 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.914357 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x"] Jul 31 22:29:26.916300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.916261 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" containerID="cri-o://f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c" gracePeriod=30 Jul 31 22:29:26.934319 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:26.934287 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:27.165703 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.165660 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:27.408006 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.407966 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:27.694542 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.694491 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn"] Jul 31 22:29:27.695099 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.695076 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" Jul 31 22:29:27.695176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.695102 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" Jul 31 22:29:27.695176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.695130 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" Jul 31 22:29:27.695176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.695138 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" Jul 31 22:29:27.695348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.695227 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" Jul 31 22:29:27.695348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.695243 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="56d12de9-97be-4410-bc31-37682fa2fe9e" containerName="istio-proxy" Jul 31 22:29:27.697449 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.697428 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.699984 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.699963 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0efd1d71-openshift-default-dockercfg-kz6cz\"" Jul 31 22:29:27.708784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.708758 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn"] Jul 31 22:29:27.719583 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.719554 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:27.733336 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733312 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-envoy\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733350 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-socket\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733380 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-credential-socket\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733553 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733482 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-certs\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733553 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733545 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-token\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733652 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733574 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istiod-ca-cert\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733652 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733622 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbvfh\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-kube-api-access-rbvfh\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733755 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733652 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-data\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.733755 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.733674 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-podinfo\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835029 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-token\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835069 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istiod-ca-cert\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835299 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835105 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbvfh\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-kube-api-access-rbvfh\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835299 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835234 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-data\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835299 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835265 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-podinfo\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835422 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835364 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-envoy\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835469 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835421 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-socket\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835473 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-credential-socket\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835503 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-certs\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835658 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835635 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-data\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835861 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835839 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-socket\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835868 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-certs\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835874 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-credential-socket\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.835953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.835888 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istiod-ca-cert\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.837648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.837629 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-envoy\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.837865 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.837845 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-podinfo\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.843525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.843504 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-token\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.843631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.843511 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbvfh\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-kube-api-access-rbvfh\") pod \"e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:27.856663 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:27.856634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:28.009210 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.009116 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:28.146814 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.146783 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn"] Jul 31 22:29:28.149168 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:29:28.149139 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca098bd3_d09f_4a2b_bfd7_5fe277e583e8.slice/crio-5ad626e5141e2d39ef96ed76eb7f7ad4b9af9d57783678b2d8c4dd0f560e5e5c WatchSource:0}: Error finding container 5ad626e5141e2d39ef96ed76eb7f7ad4b9af9d57783678b2d8c4dd0f560e5e5c: Status 404 returned error can't find the container with id 5ad626e5141e2d39ef96ed76eb7f7ad4b9af9d57783678b2d8c4dd0f560e5e5c Jul 31 22:29:28.151240 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.151206 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:28.151343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.151275 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:28.151343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.151307 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:28.735716 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.735685 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" event={"ID":"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8","Type":"ContainerStarted","Data":"bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2"} Jul 31 22:29:28.736113 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.735737 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" event={"ID":"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8","Type":"ContainerStarted","Data":"5ad626e5141e2d39ef96ed76eb7f7ad4b9af9d57783678b2d8c4dd0f560e5e5c"} Jul 31 22:29:28.761533 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:28.761485 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podStartSLOduration=1.761470804 podStartE2EDuration="1.761470804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:29:28.759894793 +0000 UTC m=+1793.324743377" watchObservedRunningTime="2026-07-31 22:29:28.761470804 +0000 UTC m=+1793.326319385" Jul 31 22:29:29.009717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:29.009634 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:29.014363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:29.014340 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:29.447379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:29.447344 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:29:29.741190 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:29.741096 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:29.742113 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:29.742093 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:30.111986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:30.111900 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:30.149898 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:30.149862 2554 patch_prober.go:28] interesting pod/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:29:30.150057 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:30.149927 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:29:30.209400 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:30.209361 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:31.242423 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:31.242390 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:31.366211 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:31.366161 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:31.732294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:31.732254 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:31.831510 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:31.831476 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:29:32.171477 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.171455 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:32.275332 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275303 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-socket\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275343 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef86f03-9f41-4934-8e40-dd62631ed258-istiod-ca-cert\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275360 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-token\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275377 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-podinfo\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275411 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-certs\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275538 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqdmj\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-kube-api-access-mqdmj\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275703 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275716 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-credential-socket\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275729 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275749 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-data\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275787 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-envoy\") pod \"2ef86f03-9f41-4934-8e40-dd62631ed258\" (UID: \"2ef86f03-9f41-4934-8e40-dd62631ed258\") " Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275830 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ef86f03-9f41-4934-8e40-dd62631ed258-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275918 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:29:32.276033 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.275964 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:29:32.276818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.276174 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.276818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.276195 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ef86f03-9f41-4934-8e40-dd62631ed258-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.276818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.276211 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.276818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.276225 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.276818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.276238 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.277873 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.277844 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:29:32.277985 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.277871 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:29:32.277985 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.277910 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-kube-api-access-mqdmj" (OuterVolumeSpecName: "kube-api-access-mqdmj") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "kube-api-access-mqdmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:29:32.278063 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.277998 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ef86f03-9f41-4934-8e40-dd62631ed258" (UID: "2ef86f03-9f41-4934-8e40-dd62631ed258"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:29:32.377711 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.377678 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.377711 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.377706 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.377711 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.377715 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ef86f03-9f41-4934-8e40-dd62631ed258-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.377931 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.377724 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqdmj\" (UniqueName: \"kubernetes.io/projected/2ef86f03-9f41-4934-8e40-dd62631ed258-kube-api-access-mqdmj\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:29:32.761498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.761458 2554 generic.go:356] "Generic (PLEG): container finished" podID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerID="f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c" exitCode=0 Jul 31 22:29:32.761724 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.761507 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" Jul 31 22:29:32.761724 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.761537 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" event={"ID":"2ef86f03-9f41-4934-8e40-dd62631ed258","Type":"ContainerDied","Data":"f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c"} Jul 31 22:29:32.761724 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.761591 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x" event={"ID":"2ef86f03-9f41-4934-8e40-dd62631ed258","Type":"ContainerDied","Data":"40e878eb34b30c7ccbcbebbad421226667a2617ac89c20dc20514a643628f822"} Jul 31 22:29:32.761724 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.761645 2554 scope.go:117] "RemoveContainer" containerID="f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c" Jul 31 22:29:32.771347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.771330 2554 scope.go:117] "RemoveContainer" containerID="a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762" Jul 31 22:29:32.782528 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.782504 2554 scope.go:117] "RemoveContainer" containerID="f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c" Jul 31 22:29:32.782924 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:29:32.782876 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c\": container with ID starting with f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c not found: ID does not exist" containerID="f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c" Jul 31 22:29:32.783001 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.782939 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c"} err="failed to get container status \"f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c\": rpc error: code = NotFound desc = could not find container \"f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c\": container with ID starting with f4248faa62448fd55d476ebc5bda33efae65e8bffbf0083eee3783227571374c not found: ID does not exist" Jul 31 22:29:32.783001 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.782964 2554 scope.go:117] "RemoveContainer" containerID="a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762" Jul 31 22:29:32.783369 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:29:32.783342 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762\": container with ID starting with a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762 not found: ID does not exist" containerID="a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762" Jul 31 22:29:32.783489 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.783373 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762"} err="failed to get container status \"a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762\": rpc error: code = NotFound desc = could not find container \"a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762\": container with ID starting with a41982179792945beb0c2402fdf4e1de0e2a2a0ed4b25f0457a9a2a1a2af8762 not found: ID does not exist" Jul 31 22:29:32.784456 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.784433 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x"] Jul 31 22:29:32.790883 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:32.790859 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-048b8d6f-gw-openshift-default-5c58c9788d-bfj8x"] Jul 31 22:29:33.957347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:33.957313 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" path="/var/lib/kubelet/pods/2ef86f03-9f41-4934-8e40-dd62631ed258/volumes" Jul 31 22:29:36.022097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:36.022068 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:29:36.023869 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:36.023848 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:29:39.141836 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:39.141794 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:29:43.831182 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:43.831084 2554 generic.go:356] "Generic (PLEG): container finished" podID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerID="bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2" exitCode=0 Jul 31 22:29:43.831182 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:43.831149 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" event={"ID":"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8","Type":"ContainerDied","Data":"bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2"} Jul 31 22:29:43.831628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:43.831530 2554 scope.go:117] "RemoveContainer" containerID="bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2" Jul 31 22:29:43.832586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:43.832552 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:43.832686 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:43.832671 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:43.832731 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:43.832718 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:29:44.741641 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:44.741590 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:44.837932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:44.837898 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" event={"ID":"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8","Type":"ContainerStarted","Data":"7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef"} Jul 31 22:29:45.009742 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:45.009650 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:45.010887 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:45.010865 2554 patch_prober.go:28] interesting pod/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 31 22:29:45.010956 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:45.010915 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 31 22:29:46.009455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:46.009419 2554 patch_prober.go:28] interesting pod/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 31 22:29:46.009885 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:46.009469 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 31 22:29:47.010303 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:47.010271 2554 patch_prober.go:28] interesting pod/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 31 22:29:47.010699 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:47.010337 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 31 22:29:48.009802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:48.009765 2554 patch_prober.go:28] interesting pod/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 31 22:29:48.009992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:48.009840 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 31 22:29:49.009902 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:49.009863 2554 patch_prober.go:28] interesting pod/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 31 22:29:49.010301 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:49.009930 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 31 22:29:50.014487 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:50.014455 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:50.014999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:50.014863 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:29:50.015550 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:29:50.015531 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:30:01.730415 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:01.730380 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:01.819024 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:01.818987 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:01.912941 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:01.912907 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:02.110974 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:02.110883 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:03.090888 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.090849 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn"] Jul 31 22:30:03.091330 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.091109 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" containerID="cri-o://7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef" gracePeriod=30 Jul 31 22:30:03.110436 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.110323 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:03.343043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.342957 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:03.573550 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.573497 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:03.854653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.854618 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm"] Jul 31 22:30:03.855186 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.855165 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" Jul 31 22:30:03.855186 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.855185 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" Jul 31 22:30:03.855375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.855280 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" Jul 31 22:30:03.855375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.855296 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" Jul 31 22:30:03.864551 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.864527 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.867192 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.867168 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a6187704-openshift-default-dockercfg-bvpnr\"" Jul 31 22:30:03.868547 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.868515 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm"] Jul 31 22:30:03.877255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.877227 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:03.971143 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971102 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-podinfo\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971337 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971152 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-socket\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971337 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971293 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-data\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971337 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971333 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-credential-socket\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-token\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971388 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-certs\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971409 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7796e31c-1702-40b9-9855-650ea9c8b6e8-istiod-ca-cert\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971457 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-envoy\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:03.971557 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:03.971497 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxlfn\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-kube-api-access-lxlfn\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.068926 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.068891 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:04.072925 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.072901 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxlfn\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-kube-api-access-lxlfn\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073030 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.072942 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-podinfo\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073030 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.072969 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-socket\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073030 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073025 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-data\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073054 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-credential-socket\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073071 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-token\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073149 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-certs\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073183 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7796e31c-1702-40b9-9855-650ea9c8b6e8-istiod-ca-cert\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073230 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-envoy\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-data\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073522 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073493 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-socket\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073571 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-credential-socket\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.073878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073669 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-certs\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.074017 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.073936 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7796e31c-1702-40b9-9855-650ea9c8b6e8-istiod-ca-cert\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.075464 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.075439 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-envoy\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.075670 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.075653 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-podinfo\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.080347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.080326 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-token\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.081166 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.081134 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxlfn\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-kube-api-access-lxlfn\") pod \"e2e-derive-a6187704-openshift-default-676f6b9768-swxqm\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.179402 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.179298 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:04.326242 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.326215 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm"] Jul 31 22:30:04.328472 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:30:04.328445 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7796e31c_1702_40b9_9855_650ea9c8b6e8.slice/crio-27caec14d18c20419c87a3ff99ab90c9076ea5b8df1c165522d7d9c66997382f WatchSource:0}: Error finding container 27caec14d18c20419c87a3ff99ab90c9076ea5b8df1c165522d7d9c66997382f: Status 404 returned error can't find the container with id 27caec14d18c20419c87a3ff99ab90c9076ea5b8df1c165522d7d9c66997382f Jul 31 22:30:04.330787 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.330755 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:30:04.330855 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.330819 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:30:04.330855 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.330847 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:30:04.633888 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.633850 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:30:04.939160 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.939128 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" event={"ID":"7796e31c-1702-40b9-9855-650ea9c8b6e8","Type":"ContainerStarted","Data":"7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24"} Jul 31 22:30:04.939347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.939177 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" event={"ID":"7796e31c-1702-40b9-9855-650ea9c8b6e8","Type":"ContainerStarted","Data":"27caec14d18c20419c87a3ff99ab90c9076ea5b8df1c165522d7d9c66997382f"} Jul 31 22:30:04.959745 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:04.959696 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podStartSLOduration=1.959678506 podStartE2EDuration="1.959678506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:30:04.956857818 +0000 UTC m=+1829.521706400" watchObservedRunningTime="2026-07-31 22:30:04.959678506 +0000 UTC m=+1829.524527087" Jul 31 22:30:05.017721 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:05.017682 2554 patch_prober.go:28] interesting pod/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:30:05.017860 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:05.017743 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:30:05.179947 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:05.179903 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:05.186145 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:05.186119 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:05.945396 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:05.945365 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:05.946459 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:05.946434 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:06.303281 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:06.303196 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:06.458116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:06.458082 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:07.556179 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:07.556140 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:07.879537 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:07.879391 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:08.021468 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.021435 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:08.118269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.118239 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:08.353392 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.353361 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:30:08.515794 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515692 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-certs\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.515950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515796 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istiod-ca-cert\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.515950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515822 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-data\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.515950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515852 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbvfh\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-kube-api-access-rbvfh\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.515950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515900 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-podinfo\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.515950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515946 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-envoy\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.516244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.515970 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-credential-socket\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.516244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516021 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:08.516244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516138 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:30:08.516244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516234 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:08.516446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516253 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:08.516446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516345 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-token\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.516446 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516422 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-socket\") pod \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\" (UID: \"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8\") " Jul 31 22:30:08.516726 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516700 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:08.516884 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516866 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.516943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516890 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.516943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516903 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.516943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516918 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.516943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.516932 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.518331 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.518305 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:30:08.518453 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.518410 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-kube-api-access-rbvfh" (OuterVolumeSpecName: "kube-api-access-rbvfh") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "kube-api-access-rbvfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:30:08.518517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.518460 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:30:08.518517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.518462 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" (UID: "ca098bd3-d09f-4a2b-bfd7-5fe277e583e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:08.618379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.618340 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbvfh\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-kube-api-access-rbvfh\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.618379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.618374 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.618379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.618387 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.618837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.618399 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:08.961725 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.961695 2554 generic.go:356] "Generic (PLEG): container finished" podID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerID="7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef" exitCode=0 Jul 31 22:30:08.961904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.961736 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" Jul 31 22:30:08.961904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.961757 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" event={"ID":"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8","Type":"ContainerDied","Data":"7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef"} Jul 31 22:30:08.961904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.961793 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn" event={"ID":"ca098bd3-d09f-4a2b-bfd7-5fe277e583e8","Type":"ContainerDied","Data":"5ad626e5141e2d39ef96ed76eb7f7ad4b9af9d57783678b2d8c4dd0f560e5e5c"} Jul 31 22:30:08.961904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.961814 2554 scope.go:117] "RemoveContainer" containerID="7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef" Jul 31 22:30:08.971384 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.971367 2554 scope.go:117] "RemoveContainer" containerID="bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2" Jul 31 22:30:08.979483 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.979464 2554 scope.go:117] "RemoveContainer" containerID="7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef" Jul 31 22:30:08.979733 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:30:08.979715 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef\": container with ID starting with 7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef not found: ID does not exist" containerID="7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef" Jul 31 22:30:08.979796 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.979740 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef"} err="failed to get container status \"7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef\": rpc error: code = NotFound desc = could not find container \"7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef\": container with ID starting with 7fa4aa9e3bcb2eca7c00d0639f0984d7ad8fb4143d173ef20359aefbb1405aef not found: ID does not exist" Jul 31 22:30:08.979796 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.979756 2554 scope.go:117] "RemoveContainer" containerID="bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2" Jul 31 22:30:08.979999 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:30:08.979979 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2\": container with ID starting with bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2 not found: ID does not exist" containerID="bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2" Jul 31 22:30:08.980056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.980007 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2"} err="failed to get container status \"bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2\": rpc error: code = NotFound desc = could not find container \"bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2\": container with ID starting with bc3fd24cc2e6ba17126753b10db73533f5399f810272428e75c4dd32ddb301a2 not found: ID does not exist" Jul 31 22:30:08.983505 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.983483 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn"] Jul 31 22:30:08.986843 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:08.986819 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0efd1d71-openshift-default-5879d76cd8-p6ldn"] Jul 31 22:30:09.959084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:09.959043 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" path="/var/lib/kubelet/pods/ca098bd3-d09f-4a2b-bfd7-5fe277e583e8/volumes" Jul 31 22:30:11.038578 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:11.038545 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:30:20.041310 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.041228 2554 generic.go:356] "Generic (PLEG): container finished" podID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerID="7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24" exitCode=0 Jul 31 22:30:20.041754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.041302 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" event={"ID":"7796e31c-1702-40b9-9855-650ea9c8b6e8","Type":"ContainerDied","Data":"7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24"} Jul 31 22:30:20.041754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.041724 2554 scope.go:117] "RemoveContainer" containerID="7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24" Jul 31 22:30:20.042789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.042758 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:30:20.042904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.042839 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:30:20.042904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.042888 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:30:20.945949 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:20.945892 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:21.048684 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:21.048647 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" event={"ID":"7796e31c-1702-40b9-9855-650ea9c8b6e8","Type":"ContainerStarted","Data":"ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac"} Jul 31 22:30:21.179862 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:21.179822 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:21.180859 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:21.180833 2554 patch_prober.go:28] interesting pod/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 31 22:30:21.181072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:21.181046 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 31 22:30:22.180114 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:22.180069 2554 patch_prober.go:28] interesting pod/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 31 22:30:22.180734 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:22.180699 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 31 22:30:23.180612 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:23.180545 2554 patch_prober.go:28] interesting pod/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 31 22:30:23.181118 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:23.180637 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 31 22:30:24.181095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:24.180681 2554 patch_prober.go:28] interesting pod/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 31 22:30:24.181095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:24.180742 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 31 22:30:25.179865 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:25.179824 2554 patch_prober.go:28] interesting pod/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 31 22:30:25.180074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:25.179889 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 31 22:30:26.183853 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:26.183822 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:26.184232 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:26.184096 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:26.184821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:26.184806 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:38.123664 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.123627 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:38.192407 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.192368 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:38.250011 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.249978 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:38.308202 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.308165 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:38.439171 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.439134 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm"] Jul 31 22:30:38.439421 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.439390 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" containerID="cri-o://ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac" gracePeriod=30 Jul 31 22:30:38.448736 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.448705 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:38.692627 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:38.692519 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:39.194166 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:39.194126 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:39.281802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:39.281768 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:39.339917 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:39.339879 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:41.135673 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:41.135632 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:30:41.186647 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:41.186594 2554 patch_prober.go:28] interesting pod/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:30:41.186806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:41.186672 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:30:43.706900 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.706878 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:43.836234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836149 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-credential-socket\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836200 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-certs\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836223 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-data\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836239 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-envoy\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836290 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxlfn\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-kube-api-access-lxlfn\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836316 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-token\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836337 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7796e31c-1702-40b9-9855-650ea9c8b6e8-istiod-ca-cert\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836381 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-socket\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836409 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-podinfo\") pod \"7796e31c-1702-40b9-9855-650ea9c8b6e8\" (UID: \"7796e31c-1702-40b9-9855-650ea9c8b6e8\") " Jul 31 22:30:43.836503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836403 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836532 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836543 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836712 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836727 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836736 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836806 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7796e31c-1702-40b9-9855-650ea9c8b6e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:30:43.836916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.836878 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:43.838536 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.838510 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-kube-api-access-lxlfn" (OuterVolumeSpecName: "kube-api-access-lxlfn") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "kube-api-access-lxlfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:30:43.838668 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.838555 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:30:43.838668 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.838555 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:30:43.838668 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.838572 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "7796e31c-1702-40b9-9855-650ea9c8b6e8" (UID: "7796e31c-1702-40b9-9855-650ea9c8b6e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:30:43.937353 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.937320 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxlfn\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-kube-api-access-lxlfn\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.937353 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.937349 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.937353 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.937358 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7796e31c-1702-40b9-9855-650ea9c8b6e8-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.937583 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.937367 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.937583 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.937377 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:43.937583 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:43.937386 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7796e31c-1702-40b9-9855-650ea9c8b6e8-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:30:44.169294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.169223 2554 generic.go:356] "Generic (PLEG): container finished" podID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerID="ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac" exitCode=0 Jul 31 22:30:44.169294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.169272 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" Jul 31 22:30:44.169463 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.169293 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" event={"ID":"7796e31c-1702-40b9-9855-650ea9c8b6e8","Type":"ContainerDied","Data":"ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac"} Jul 31 22:30:44.169463 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.169337 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm" event={"ID":"7796e31c-1702-40b9-9855-650ea9c8b6e8","Type":"ContainerDied","Data":"27caec14d18c20419c87a3ff99ab90c9076ea5b8df1c165522d7d9c66997382f"} Jul 31 22:30:44.169463 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.169359 2554 scope.go:117] "RemoveContainer" containerID="ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac" Jul 31 22:30:44.178148 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.178131 2554 scope.go:117] "RemoveContainer" containerID="7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24" Jul 31 22:30:44.186093 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.186073 2554 scope.go:117] "RemoveContainer" containerID="ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac" Jul 31 22:30:44.186327 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.186306 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm"] Jul 31 22:30:44.186389 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:30:44.186327 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac\": container with ID starting with ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac not found: ID does not exist" containerID="ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac" Jul 31 22:30:44.186389 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.186351 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac"} err="failed to get container status \"ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac\": rpc error: code = NotFound desc = could not find container \"ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac\": container with ID starting with ed0a41634123d3ba14a5614f7a2d247997922e3c6d33ebfa9d78c694558081ac not found: ID does not exist" Jul 31 22:30:44.186389 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.186367 2554 scope.go:117] "RemoveContainer" containerID="7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24" Jul 31 22:30:44.186611 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:30:44.186580 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24\": container with ID starting with 7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24 not found: ID does not exist" containerID="7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24" Jul 31 22:30:44.186667 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.186627 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24"} err="failed to get container status \"7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24\": rpc error: code = NotFound desc = could not find container \"7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24\": container with ID starting with 7aa2020200aec42ce54df8f8b69082ef21330b754e5aa6b5240df4f97b16ef24 not found: ID does not exist" Jul 31 22:30:44.189774 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:44.189752 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a6187704-openshift-default-676f6b9768-swxqm"] Jul 31 22:30:45.958688 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:45.958646 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" path="/var/lib/kubelet/pods/7796e31c-1702-40b9-9855-650ea9c8b6e8/volumes" Jul 31 22:30:51.113619 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:51.113569 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:52.537643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:52.537593 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:30:57.296184 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:57.296144 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:57.389230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:57.389188 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:57.479838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:57.479808 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:57.579743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:57.579655 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:30:58.042578 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:30:58.042542 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:03.769818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:03.769786 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:24.039436 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:24.039397 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:29.967038 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:29.967003 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:31.186807 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:31.186763 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:46.107912 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:46.107875 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:51.861775 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:51.861736 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:51.971808 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:51.971768 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:31:52.067134 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:31:52.067100 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:02.617907 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:02.617869 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:13.678488 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:13.678447 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:19.704825 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:19.704792 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:19.775385 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:19.775338 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:19.867918 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:19.867876 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:19.964381 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:19.964291 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:20.074960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:20.074922 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:20.787073 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:20.787034 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:20.882797 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:20.882759 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:21.248008 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:21.247967 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:32:22.492076 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:22.492035 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:28.614718 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:28.614626 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:28.715581 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:28.715543 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:29.302140 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:29.302107 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:29.841856 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:29.841820 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:32:35.133822 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.133787 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:35.230156 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.230088 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:35.385254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.385167 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:35.678474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.678390 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q"] Jul 31 22:32:35.678970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.678937 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" Jul 31 22:32:35.678970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.678962 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.678979 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.678987 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef86f03-9f41-4934-8e40-dd62631ed258" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.678996 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679005 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679015 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679023 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679039 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679047 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679167 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679181 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca098bd3-d09f-4a2b-bfd7-5fe277e583e8" containerName="istio-proxy" Jul 31 22:32:35.679187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.679189 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" Jul 31 22:32:35.682501 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.682479 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.684911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.684890 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d8ee576a-openshift-default-dockercfg-fl48s\"" Jul 31 22:32:35.702708 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.702685 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q"] Jul 31 22:32:35.707817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.707792 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:35.795471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795437 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-data\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795468 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21253474-42e6-4327-8db5-d40e2478290f-istio-podinfo\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795497 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-envoy\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795533 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-credential-socket\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795553 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-istio-token\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795616 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-socket\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795640 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-certs\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795663 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21253474-42e6-4327-8db5-d40e2478290f-istiod-ca-cert\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.795701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.795679 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4cs\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-kube-api-access-7p4cs\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.801978 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.801950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:35.896840 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896797 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-data\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.896840 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896843 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21253474-42e6-4327-8db5-d40e2478290f-istio-podinfo\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897051 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896865 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-envoy\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897051 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896883 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-credential-socket\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897051 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-istio-token\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897051 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896953 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-socket\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897051 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.896982 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-certs\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897051 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897015 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21253474-42e6-4327-8db5-d40e2478290f-istiod-ca-cert\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897125 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4cs\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-kube-api-access-7p4cs\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897211 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-data\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897253 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-credential-socket\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897740 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897715 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-certs\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897866 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897722 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-socket\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.897866 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.897770 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21253474-42e6-4327-8db5-d40e2478290f-istiod-ca-cert\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.899290 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.899270 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-envoy\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.899387 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.899369 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21253474-42e6-4327-8db5-d40e2478290f-istio-podinfo\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.904525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.904499 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4cs\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-kube-api-access-7p4cs\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.904850 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.904823 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-istio-token\") pod \"e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:35.996966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:35.996937 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d8ee576a-openshift-default-dockercfg-fl48s\"" Jul 31 22:32:36.005729 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.005705 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:36.152442 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.152391 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q"] Jul 31 22:32:36.154676 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:32:36.154652 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21253474_42e6_4327_8db5_d40e2478290f.slice/crio-7004644f8baa3b199fcd9d0b92d865e968e79fad1235d2eb5b07c51e74c06b8f WatchSource:0}: Error finding container 7004644f8baa3b199fcd9d0b92d865e968e79fad1235d2eb5b07c51e74c06b8f: Status 404 returned error can't find the container with id 7004644f8baa3b199fcd9d0b92d865e968e79fad1235d2eb5b07c51e74c06b8f Jul 31 22:32:36.156731 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.156699 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:32:36.156823 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.156762 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:32:36.156823 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.156794 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:32:36.729520 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.729482 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" event={"ID":"21253474-42e6-4327-8db5-d40e2478290f","Type":"ContainerStarted","Data":"2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6"} Jul 31 22:32:36.729717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.729541 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" event={"ID":"21253474-42e6-4327-8db5-d40e2478290f","Type":"ContainerStarted","Data":"7004644f8baa3b199fcd9d0b92d865e968e79fad1235d2eb5b07c51e74c06b8f"} Jul 31 22:32:36.749450 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:36.749408 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podStartSLOduration=1.749393507 podStartE2EDuration="1.749393507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:32:36.74738404 +0000 UTC m=+1981.312232623" watchObservedRunningTime="2026-07-31 22:32:36.749393507 +0000 UTC m=+1981.314242088" Jul 31 22:32:37.006721 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:37.006645 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:37.011778 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:37.011753 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:37.735779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:37.735748 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:37.736876 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:37.736852 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:38.710338 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:38.710286 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:38.805881 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:38.805846 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:39.838250 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:39.838214 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:40.002983 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:40.002945 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:40.336357 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:40.336326 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:40.433127 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:40.433095 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:42.241743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:42.241702 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:32:44.261685 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:44.261644 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:52.737272 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.737235 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 31 22:32:52.737762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.737303 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 31 22:32:52.826619 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.826565 2554 generic.go:356] "Generic (PLEG): container finished" podID="21253474-42e6-4327-8db5-d40e2478290f" containerID="2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6" exitCode=0 Jul 31 22:32:52.826776 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.826628 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" event={"ID":"21253474-42e6-4327-8db5-d40e2478290f","Type":"ContainerDied","Data":"2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6"} Jul 31 22:32:52.827042 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.827026 2554 scope.go:117] "RemoveContainer" containerID="2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6" Jul 31 22:32:52.828065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.828037 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:32:52.828116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.828105 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:32:52.828152 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:52.828135 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:32:53.833072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:53.833035 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" event={"ID":"21253474-42e6-4327-8db5-d40e2478290f","Type":"ContainerStarted","Data":"996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc"} Jul 31 22:32:54.006270 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:54.006216 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:54.006548 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:54.006524 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 31 22:32:54.006627 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:54.006585 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 31 22:32:55.006366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:55.006334 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 31 22:32:55.006779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:55.006386 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 31 22:32:56.006729 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:56.006694 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 31 22:32:56.007098 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:56.006740 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 31 22:32:57.006956 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:57.006918 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 31 22:32:57.007352 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:57.006990 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 31 22:32:58.006409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:58.006370 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 31 22:32:58.006655 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:58.006451 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 31 22:32:59.009947 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:59.009919 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:59.010317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:59.010254 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:59.011040 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:59.011020 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:32:59.684191 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:59.684154 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:32:59.785153 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:32:59.785122 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:03.768063 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:03.768028 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:04.431592 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:04.431550 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:04.532627 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:04.532576 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:04.881154 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:04.881114 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:10.674714 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:10.674679 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:10.729400 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:10.729366 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:15.490209 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:15.490174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:21.480124 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:21.480074 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:21.547679 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:21.547640 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:21.608519 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:21.608474 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:21.665467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:21.665431 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:22.050913 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:22.050877 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:23.443266 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:23.443231 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:33:27.752867 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:27.752825 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:27.810917 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:27.810873 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:28.006563 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.006485 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q"] Jul 31 22:33:28.006785 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.006757 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" containerID="cri-o://996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc" gracePeriod=30 Jul 31 22:33:28.026711 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.026673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:28.272980 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.272873 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:28.388815 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.388776 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:28.489122 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.489080 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:28.607340 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:28.607249 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:29.012508 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:29.012472 2554 patch_prober.go:28] interesting pod/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:33:29.012902 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:29.012534 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:33:33.258892 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.258866 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:33:33.434252 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434225 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p4cs\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-kube-api-access-7p4cs\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434304 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-istio-token\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434338 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-envoy\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434358 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-socket\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434409 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434385 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21253474-42e6-4327-8db5-d40e2478290f-istiod-ca-cert\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434587 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434480 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-certs\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434587 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434551 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-data\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434587 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434577 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21253474-42e6-4327-8db5-d40e2478290f-istio-podinfo\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434776 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434655 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-credential-socket\") pod \"21253474-42e6-4327-8db5-d40e2478290f\" (UID: \"21253474-42e6-4327-8db5-d40e2478290f\") " Jul 31 22:33:33.434776 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434739 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:33:33.434880 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434806 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:33:33.434880 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434840 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21253474-42e6-4327-8db5-d40e2478290f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:33:33.434993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434944 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:33:33.434993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.434974 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-data" (OuterVolumeSpecName: "istio-data") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:33:33.435113 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.435103 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.435164 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.435117 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.435164 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.435130 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21253474-42e6-4327-8db5-d40e2478290f-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.435164 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.435143 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.435164 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.435155 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.436697 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.436672 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-istio-token" (OuterVolumeSpecName: "istio-token") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:33:33.436807 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.436797 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21253474-42e6-4327-8db5-d40e2478290f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:33:33.437017 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.436996 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:33:33.437068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.437042 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-kube-api-access-7p4cs" (OuterVolumeSpecName: "kube-api-access-7p4cs") pod "21253474-42e6-4327-8db5-d40e2478290f" (UID: "21253474-42e6-4327-8db5-d40e2478290f"). InnerVolumeSpecName "kube-api-access-7p4cs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:33:33.536000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.535967 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21253474-42e6-4327-8db5-d40e2478290f-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.536000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.535994 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21253474-42e6-4327-8db5-d40e2478290f-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.536000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.536003 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p4cs\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-kube-api-access-7p4cs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:33.536000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:33.536012 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21253474-42e6-4327-8db5-d40e2478290f-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:33:34.026996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.026958 2554 generic.go:356] "Generic (PLEG): container finished" podID="21253474-42e6-4327-8db5-d40e2478290f" containerID="996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc" exitCode=0 Jul 31 22:33:34.027163 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.027011 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" Jul 31 22:33:34.027163 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.027009 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" event={"ID":"21253474-42e6-4327-8db5-d40e2478290f","Type":"ContainerDied","Data":"996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc"} Jul 31 22:33:34.027163 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.027131 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q" event={"ID":"21253474-42e6-4327-8db5-d40e2478290f","Type":"ContainerDied","Data":"7004644f8baa3b199fcd9d0b92d865e968e79fad1235d2eb5b07c51e74c06b8f"} Jul 31 22:33:34.027163 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.027161 2554 scope.go:117] "RemoveContainer" containerID="996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc" Jul 31 22:33:34.036137 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.036120 2554 scope.go:117] "RemoveContainer" containerID="2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6" Jul 31 22:33:34.044309 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.044291 2554 scope.go:117] "RemoveContainer" containerID="996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc" Jul 31 22:33:34.044416 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.044393 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q"] Jul 31 22:33:34.044555 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:33:34.044539 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc\": container with ID starting with 996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc not found: ID does not exist" containerID="996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc" Jul 31 22:33:34.044620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.044563 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc"} err="failed to get container status \"996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc\": rpc error: code = NotFound desc = could not find container \"996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc\": container with ID starting with 996aee3ab002380565f04b7129a227d3c6945e8916d0e4065c30c9d624d882dc not found: ID does not exist" Jul 31 22:33:34.044620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.044581 2554 scope.go:117] "RemoveContainer" containerID="2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6" Jul 31 22:33:34.044833 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:33:34.044816 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6\": container with ID starting with 2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6 not found: ID does not exist" containerID="2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6" Jul 31 22:33:34.044885 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.044838 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6"} err="failed to get container status \"2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6\": rpc error: code = NotFound desc = could not find container \"2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6\": container with ID starting with 2ba8fc6a7655072b5905674a3c711ebe750d32dc9d4df012ac1f5929abba16a6 not found: ID does not exist" Jul 31 22:33:34.051957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.051934 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d8ee576a-openshift-default-6965dc8496-rhh7q"] Jul 31 22:33:34.425011 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.424925 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:34.487290 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.487254 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:34.558456 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.558424 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:34.659374 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.659341 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:34.838754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:34.838715 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:35.957231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:35.957194 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21253474-42e6-4327-8db5-d40e2478290f" path="/var/lib/kubelet/pods/21253474-42e6-4327-8db5-d40e2478290f/volumes" Jul 31 22:33:40.545970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:40.545935 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:40.604343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:40.604311 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:33:59.883294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:33:59.883213 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:34:09.157072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.157022 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h"] Jul 31 22:34:09.157578 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.157350 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" podUID="2c12ec64-59eb-478d-9c86-df289b4004a8" containerName="manager" containerID="cri-o://618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292" gracePeriod=10 Jul 31 22:34:09.409766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.409702 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:34:09.562580 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.562545 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c12ec64-59eb-478d-9c86-df289b4004a8-extensions-socket-volume\") pod \"2c12ec64-59eb-478d-9c86-df289b4004a8\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " Jul 31 22:34:09.562784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.562657 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnkqr\" (UniqueName: \"kubernetes.io/projected/2c12ec64-59eb-478d-9c86-df289b4004a8-kube-api-access-fnkqr\") pod \"2c12ec64-59eb-478d-9c86-df289b4004a8\" (UID: \"2c12ec64-59eb-478d-9c86-df289b4004a8\") " Jul 31 22:34:09.562997 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.562974 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c12ec64-59eb-478d-9c86-df289b4004a8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2c12ec64-59eb-478d-9c86-df289b4004a8" (UID: "2c12ec64-59eb-478d-9c86-df289b4004a8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:34:09.564791 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.564763 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c12ec64-59eb-478d-9c86-df289b4004a8-kube-api-access-fnkqr" (OuterVolumeSpecName: "kube-api-access-fnkqr") pod "2c12ec64-59eb-478d-9c86-df289b4004a8" (UID: "2c12ec64-59eb-478d-9c86-df289b4004a8"). InnerVolumeSpecName "kube-api-access-fnkqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:34:09.664095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.663999 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnkqr\" (UniqueName: \"kubernetes.io/projected/2c12ec64-59eb-478d-9c86-df289b4004a8-kube-api-access-fnkqr\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:34:09.664095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:09.664035 2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c12ec64-59eb-478d-9c86-df289b4004a8-extensions-socket-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:34:10.204486 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.204451 2554 generic.go:356] "Generic (PLEG): container finished" podID="2c12ec64-59eb-478d-9c86-df289b4004a8" containerID="618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292" exitCode=0 Jul 31 22:34:10.204957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.204492 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" Jul 31 22:34:10.204957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.204524 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" event={"ID":"2c12ec64-59eb-478d-9c86-df289b4004a8","Type":"ContainerDied","Data":"618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292"} Jul 31 22:34:10.204957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.204576 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h" event={"ID":"2c12ec64-59eb-478d-9c86-df289b4004a8","Type":"ContainerDied","Data":"c385ccfb516c65bcf48ca38073005315e3366e8a86629324b1891f518982737b"} Jul 31 22:34:10.204957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.204637 2554 scope.go:117] "RemoveContainer" containerID="618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292" Jul 31 22:34:10.214588 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.214565 2554 scope.go:117] "RemoveContainer" containerID="618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292" Jul 31 22:34:10.214844 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:34:10.214822 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292\": container with ID starting with 618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292 not found: ID does not exist" containerID="618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292" Jul 31 22:34:10.214908 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.214856 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292"} err="failed to get container status \"618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292\": rpc error: code = NotFound desc = could not find container \"618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292\": container with ID starting with 618b5cb686fadc38de11301a74b1568329bc10e30978dc9593f719c5411b3292 not found: ID does not exist" Jul 31 22:34:10.222060 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.222040 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h"] Jul 31 22:34:10.226306 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.226286 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gq89h"] Jul 31 22:34:10.782852 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.782806 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:34:10.837578 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:10.837541 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:34:11.957216 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:11.957178 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c12ec64-59eb-478d-9c86-df289b4004a8" path="/var/lib/kubelet/pods/2c12ec64-59eb-478d-9c86-df289b4004a8/volumes" Jul 31 22:34:26.443658 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:26.443620 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:34:36.053451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:36.053419 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:34:36.056957 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:34:36.056935 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:35:24.799108 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799070 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl"] Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799435 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c12ec64-59eb-478d-9c86-df289b4004a8" containerName="manager" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799446 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c12ec64-59eb-478d-9c86-df289b4004a8" containerName="manager" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799460 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799467 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799477 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799482 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799547 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c12ec64-59eb-478d-9c86-df289b4004a8" containerName="manager" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799554 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" Jul 31 22:35:24.799643 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.799565 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7796e31c-1702-40b9-9855-650ea9c8b6e8" containerName="istio-proxy" Jul 31 22:35:24.802579 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.802563 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:24.805350 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.805329 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2sxtx\"" Jul 31 22:35:24.816222 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.816196 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl"] Jul 31 22:35:24.835252 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.835224 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35cc624c-c12c-4dab-b550-64f61496176d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-rgbzl\" (UID: \"35cc624c-c12c-4dab-b550-64f61496176d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:24.835368 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.835263 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzqzf\" (UniqueName: \"kubernetes.io/projected/35cc624c-c12c-4dab-b550-64f61496176d-kube-api-access-kzqzf\") pod \"kuadrant-operator-controller-manager-68cf589589-rgbzl\" (UID: \"35cc624c-c12c-4dab-b550-64f61496176d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:24.936229 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.936195 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35cc624c-c12c-4dab-b550-64f61496176d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-rgbzl\" (UID: \"35cc624c-c12c-4dab-b550-64f61496176d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:24.936367 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.936241 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzqzf\" (UniqueName: \"kubernetes.io/projected/35cc624c-c12c-4dab-b550-64f61496176d-kube-api-access-kzqzf\") pod \"kuadrant-operator-controller-manager-68cf589589-rgbzl\" (UID: \"35cc624c-c12c-4dab-b550-64f61496176d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:24.936674 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.936655 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35cc624c-c12c-4dab-b550-64f61496176d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-rgbzl\" (UID: \"35cc624c-c12c-4dab-b550-64f61496176d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:24.945101 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:24.945079 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzqzf\" (UniqueName: \"kubernetes.io/projected/35cc624c-c12c-4dab-b550-64f61496176d-kube-api-access-kzqzf\") pod \"kuadrant-operator-controller-manager-68cf589589-rgbzl\" (UID: \"35cc624c-c12c-4dab-b550-64f61496176d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:25.112996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:25.112917 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:25.251184 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:25.251159 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl"] Jul 31 22:35:25.253585 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:35:25.253559 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35cc624c_c12c_4dab_b550_64f61496176d.slice/crio-b4734556eaf1b1154f8e691d799b1b458b537cda33a2de5c0596ae62e064bacc WatchSource:0}: Error finding container b4734556eaf1b1154f8e691d799b1b458b537cda33a2de5c0596ae62e064bacc: Status 404 returned error can't find the container with id b4734556eaf1b1154f8e691d799b1b458b537cda33a2de5c0596ae62e064bacc Jul 31 22:35:25.567011 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:25.566979 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" event={"ID":"35cc624c-c12c-4dab-b550-64f61496176d","Type":"ContainerStarted","Data":"92fa1acdbd9183b577d0b386b5f9a904076d84ab5de883b02e2391099e608b84"} Jul 31 22:35:25.567213 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:25.567038 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" event={"ID":"35cc624c-c12c-4dab-b550-64f61496176d","Type":"ContainerStarted","Data":"b4734556eaf1b1154f8e691d799b1b458b537cda33a2de5c0596ae62e064bacc"} Jul 31 22:35:25.567213 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:25.567161 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:25.588582 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:25.588532 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" podStartSLOduration=1.588517697 podStartE2EDuration="1.588517697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:35:25.586094142 +0000 UTC m=+2150.150942730" watchObservedRunningTime="2026-07-31 22:35:25.588517697 +0000 UTC m=+2150.153366279" Jul 31 22:35:36.574050 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:36.573975 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-rgbzl" Jul 31 22:35:43.189566 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.189529 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:43.476569 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.476489 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn"] Jul 31 22:35:43.476974 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.476961 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="21253474-42e6-4327-8db5-d40e2478290f" containerName="istio-proxy" Jul 31 22:35:43.479987 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.479964 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.483988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.483957 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9c96e5e7-openshift-default-dockercfg-fkz2r\"" Jul 31 22:35:43.493395 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.493363 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn"] Jul 31 22:35:43.506775 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.506739 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:43.605313 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605279 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-certs\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605313 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605322 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-envoy\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605362 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-socket\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605424 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80105f3c-7475-476e-8bd1-4df4751b40b8-istiod-ca-cert\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605473 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-token\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605492 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-credential-socket\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605514 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-data\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605634 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605535 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-podinfo\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.605894 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.605680 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnfr4\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-kube-api-access-wnfr4\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706396 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-socket\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706409 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80105f3c-7475-476e-8bd1-4df4751b40b8-istiod-ca-cert\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706522 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-token\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706555 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-credential-socket\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706755 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706585 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-data\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706815 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706776 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-podinfo\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706877 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706852 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-socket\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706936 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706920 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnfr4\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-kube-api-access-wnfr4\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706941 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-credential-socket\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.706992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.706972 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-certs\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.707102 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.707000 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-envoy\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.707258 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.707235 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-certs\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.707258 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.707248 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80105f3c-7475-476e-8bd1-4df4751b40b8-istiod-ca-cert\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.707385 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.707333 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-data\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.709141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.709112 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-envoy\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.709360 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.709342 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-podinfo\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.714162 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.714138 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-token\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.714784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.714763 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnfr4\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-kube-api-access-wnfr4\") pod \"e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.792717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.792647 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:43.942021 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.941994 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn"] Jul 31 22:35:43.944101 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:35:43.944075 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80105f3c_7475_476e_8bd1_4df4751b40b8.slice/crio-2982856187c15b7fb894973333c770defb5af53aa3fea4ca841a06b5378f8481 WatchSource:0}: Error finding container 2982856187c15b7fb894973333c770defb5af53aa3fea4ca841a06b5378f8481: Status 404 returned error can't find the container with id 2982856187c15b7fb894973333c770defb5af53aa3fea4ca841a06b5378f8481 Jul 31 22:35:43.946351 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.946316 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:35:43.946455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.946394 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:35:43.946455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.946439 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:35:43.974544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:43.974517 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:44.659728 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:44.659690 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" event={"ID":"80105f3c-7475-476e-8bd1-4df4751b40b8","Type":"ContainerStarted","Data":"3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b"} Jul 31 22:35:44.660118 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:44.659753 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" event={"ID":"80105f3c-7475-476e-8bd1-4df4751b40b8","Type":"ContainerStarted","Data":"2982856187c15b7fb894973333c770defb5af53aa3fea4ca841a06b5378f8481"} Jul 31 22:35:44.680448 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:44.680396 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podStartSLOduration=1.6803661349999999 podStartE2EDuration="1.680366135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:35:44.679392002 +0000 UTC m=+2169.244240613" watchObservedRunningTime="2026-07-31 22:35:44.680366135 +0000 UTC m=+2169.245214719" Jul 31 22:35:44.792881 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:44.792838 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:45.798169 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:45.798137 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:46.638972 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:46.638931 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:46.670410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:46.670382 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:46.671238 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:46.671220 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:35:46.738293 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:46.738258 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:46.839624 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:46.839570 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:35:47.786133 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:47.786098 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:47.933074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:47.933035 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:48.316712 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:48.316667 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:48.421728 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:48.421693 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:35:54.638907 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:35:54.638872 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:36:00.750023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:00.749992 2554 generic.go:356] "Generic (PLEG): container finished" podID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerID="3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b" exitCode=0 Jul 31 22:36:00.750451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:00.750067 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" event={"ID":"80105f3c-7475-476e-8bd1-4df4751b40b8","Type":"ContainerDied","Data":"3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b"} Jul 31 22:36:00.750499 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:00.750472 2554 scope.go:117] "RemoveContainer" containerID="3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b" Jul 31 22:36:00.751517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:00.751489 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:36:00.751587 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:00.751553 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:36:00.751653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:00.751611 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:36:01.671003 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:01.670965 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:36:01.757281 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:01.757243 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" event={"ID":"80105f3c-7475-476e-8bd1-4df4751b40b8","Type":"ContainerStarted","Data":"3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c"} Jul 31 22:36:01.793278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:01.793237 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:36:01.794979 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:01.794946 2554 patch_prober.go:28] interesting pod/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 31 22:36:01.795231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:01.795200 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 31 22:36:02.793224 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:02.793179 2554 patch_prober.go:28] interesting pod/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 31 22:36:02.793752 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:02.793247 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 31 22:36:03.793324 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:03.793282 2554 patch_prober.go:28] interesting pod/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 31 22:36:03.793970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:03.793933 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 31 22:36:04.793709 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:04.793664 2554 patch_prober.go:28] interesting pod/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 31 22:36:04.794230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:04.793747 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 31 22:36:05.794062 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:05.794012 2554 patch_prober.go:28] interesting pod/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 31 22:36:05.795996 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:05.794102 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 31 22:36:06.797432 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:06.797391 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:36:06.797924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:06.797816 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:36:06.798428 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:06.798409 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:36:11.690287 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:11.690251 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:36:15.135095 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.135059 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt"] Jul 31 22:36:15.137320 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.137300 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.139955 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.139934 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9c96e5e7\"/\"openshift-service-ca.crt\"" Jul 31 22:36:15.140768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.140748 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9c96e5e7\"/\"e2e-lifecycle-model-9c96e5e7-kserve-self-signed-certs\"" Jul 31 22:36:15.140906 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.140769 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9c96e5e7\"/\"default-dockercfg-vjf97\"" Jul 31 22:36:15.140906 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.140825 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9c96e5e7\"/\"kube-root-ca.crt\"" Jul 31 22:36:15.147478 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.147454 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt"] Jul 31 22:36:15.165144 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.165112 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:36:15.203267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.203230 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-tmp-dir\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.203267 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.203265 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjrhv\" (UniqueName: \"kubernetes.io/projected/434b4176-d5fc-406f-b39a-bf39ee11d67d-kube-api-access-vjrhv\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.203471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.203290 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-dshm\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.203471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.203399 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-model-cache\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.203551 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.203502 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-home\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.203551 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.203526 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434b4176-d5fc-406f-b39a-bf39ee11d67d-tls-certs\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.304760 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.304724 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-home\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.304953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.304763 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434b4176-d5fc-406f-b39a-bf39ee11d67d-tls-certs\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.304953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.304798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-tmp-dir\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.305337 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.305309 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-tmp-dir\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.305514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.305352 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-home\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.305640 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.305439 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjrhv\" (UniqueName: \"kubernetes.io/projected/434b4176-d5fc-406f-b39a-bf39ee11d67d-kube-api-access-vjrhv\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.305640 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.305556 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-dshm\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.305757 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.305641 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-model-cache\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.306024 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.305983 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-model-cache\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.307570 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.307546 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-dshm\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.307683 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.307557 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434b4176-d5fc-406f-b39a-bf39ee11d67d-tls-certs\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.312945 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.312833 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjrhv\" (UniqueName: \"kubernetes.io/projected/434b4176-d5fc-406f-b39a-bf39ee11d67d-kube-api-access-vjrhv\") pod \"e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.449092 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.449059 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:15.583436 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.583414 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt"] Jul 31 22:36:15.585759 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:36:15.585726 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod434b4176_d5fc_406f_b39a_bf39ee11d67d.slice/crio-8ce070f7c2951b3a1dd798f6d46f33f14899ca52779ea36565a6e483ab715253 WatchSource:0}: Error finding container 8ce070f7c2951b3a1dd798f6d46f33f14899ca52779ea36565a6e483ab715253: Status 404 returned error can't find the container with id 8ce070f7c2951b3a1dd798f6d46f33f14899ca52779ea36565a6e483ab715253 Jul 31 22:36:15.828681 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.828560 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" event={"ID":"434b4176-d5fc-406f-b39a-bf39ee11d67d","Type":"ContainerStarted","Data":"d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6"} Jul 31 22:36:15.828681 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.828627 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" event={"ID":"434b4176-d5fc-406f-b39a-bf39ee11d67d","Type":"ContainerStarted","Data":"8ce070f7c2951b3a1dd798f6d46f33f14899ca52779ea36565a6e483ab715253"} Jul 31 22:36:15.844436 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:15.844378 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" podStartSLOduration=0.844363654 podStartE2EDuration="844.363654ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:36:15.843594581 +0000 UTC m=+2200.408443154" watchObservedRunningTime="2026-07-31 22:36:15.844363654 +0000 UTC m=+2200.409212235" Jul 31 22:36:17.444522 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:17.444483 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:36:25.449872 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:25.449833 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:25.450265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:25.449919 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:25.462466 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:25.462445 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:25.890922 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:25.890840 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:36:30.339271 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:30.339235 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:36:53.941610 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:53.941556 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:36:56.580906 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:56.580866 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:36:57.797948 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:57.797913 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:36:58.840891 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:36:58.840849 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:37:00.110405 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:00.110370 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:37:05.098636 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:05.098588 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt"] Jul 31 22:37:05.142290 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:05.142254 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:37:20.117986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.117920 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" podUID="434b4176-d5fc-406f-b39a-bf39ee11d67d" containerName="main" containerID="cri-o://d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6" gracePeriod=45 Jul 31 22:37:20.367323 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.367297 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:37:20.506826 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.506797 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-dshm\") pod \"434b4176-d5fc-406f-b39a-bf39ee11d67d\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " Jul 31 22:37:20.507016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.506857 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjrhv\" (UniqueName: \"kubernetes.io/projected/434b4176-d5fc-406f-b39a-bf39ee11d67d-kube-api-access-vjrhv\") pod \"434b4176-d5fc-406f-b39a-bf39ee11d67d\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " Jul 31 22:37:20.507016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.506913 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-tmp-dir\") pod \"434b4176-d5fc-406f-b39a-bf39ee11d67d\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " Jul 31 22:37:20.507016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.506933 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-home\") pod \"434b4176-d5fc-406f-b39a-bf39ee11d67d\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " Jul 31 22:37:20.507016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.506956 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434b4176-d5fc-406f-b39a-bf39ee11d67d-tls-certs\") pod \"434b4176-d5fc-406f-b39a-bf39ee11d67d\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " Jul 31 22:37:20.507016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.506991 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-model-cache\") pod \"434b4176-d5fc-406f-b39a-bf39ee11d67d\" (UID: \"434b4176-d5fc-406f-b39a-bf39ee11d67d\") " Jul 31 22:37:20.507289 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.507238 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-home" (OuterVolumeSpecName: "home") pod "434b4176-d5fc-406f-b39a-bf39ee11d67d" (UID: "434b4176-d5fc-406f-b39a-bf39ee11d67d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:37:20.507351 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.507244 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "434b4176-d5fc-406f-b39a-bf39ee11d67d" (UID: "434b4176-d5fc-406f-b39a-bf39ee11d67d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:37:20.507412 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.507354 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-model-cache" (OuterVolumeSpecName: "model-cache") pod "434b4176-d5fc-406f-b39a-bf39ee11d67d" (UID: "434b4176-d5fc-406f-b39a-bf39ee11d67d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:37:20.509185 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.509159 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-dshm" (OuterVolumeSpecName: "dshm") pod "434b4176-d5fc-406f-b39a-bf39ee11d67d" (UID: "434b4176-d5fc-406f-b39a-bf39ee11d67d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:37:20.509261 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.509222 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434b4176-d5fc-406f-b39a-bf39ee11d67d-kube-api-access-vjrhv" (OuterVolumeSpecName: "kube-api-access-vjrhv") pod "434b4176-d5fc-406f-b39a-bf39ee11d67d" (UID: "434b4176-d5fc-406f-b39a-bf39ee11d67d"). InnerVolumeSpecName "kube-api-access-vjrhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:37:20.509316 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.509295 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/434b4176-d5fc-406f-b39a-bf39ee11d67d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "434b4176-d5fc-406f-b39a-bf39ee11d67d" (UID: "434b4176-d5fc-406f-b39a-bf39ee11d67d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 31 22:37:20.607897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.607861 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-dshm\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:37:20.607897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.607892 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjrhv\" (UniqueName: \"kubernetes.io/projected/434b4176-d5fc-406f-b39a-bf39ee11d67d-kube-api-access-vjrhv\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:37:20.607897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.607902 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-tmp-dir\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:37:20.608130 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.607913 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-home\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:37:20.608130 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.607920 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/434b4176-d5fc-406f-b39a-bf39ee11d67d-tls-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:37:20.608130 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:20.607928 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/434b4176-d5fc-406f-b39a-bf39ee11d67d-model-cache\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:37:21.162335 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.162295 2554 generic.go:356] "Generic (PLEG): container finished" podID="434b4176-d5fc-406f-b39a-bf39ee11d67d" containerID="d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6" exitCode=0 Jul 31 22:37:21.162335 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.162333 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" Jul 31 22:37:21.162817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.162334 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" event={"ID":"434b4176-d5fc-406f-b39a-bf39ee11d67d","Type":"ContainerDied","Data":"d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6"} Jul 31 22:37:21.162817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.162453 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt" event={"ID":"434b4176-d5fc-406f-b39a-bf39ee11d67d","Type":"ContainerDied","Data":"8ce070f7c2951b3a1dd798f6d46f33f14899ca52779ea36565a6e483ab715253"} Jul 31 22:37:21.162817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.162472 2554 scope.go:117] "RemoveContainer" containerID="d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6" Jul 31 22:37:21.171322 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.171304 2554 scope.go:117] "RemoveContainer" containerID="d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6" Jul 31 22:37:21.171555 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:37:21.171539 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6\": container with ID starting with d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6 not found: ID does not exist" containerID="d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6" Jul 31 22:37:21.171626 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.171560 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6"} err="failed to get container status \"d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6\": rpc error: code = NotFound desc = could not find container \"d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6\": container with ID starting with d2f70240c8a208b9e31f9d098da616c978e2fd2df180a05c04561269de0713e6 not found: ID does not exist" Jul 31 22:37:21.184570 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.184546 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt"] Jul 31 22:37:21.187941 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.187921 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9c96e5e7/e2e-lifecycle-model-9c96e5e7-kserve-65f5884c59-6qwrt"] Jul 31 22:37:21.958210 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:21.958168 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="434b4176-d5fc-406f-b39a-bf39ee11d67d" path="/var/lib/kubelet/pods/434b4176-d5fc-406f-b39a-bf39ee11d67d/volumes" Jul 31 22:37:41.455043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:41.455006 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:37:42.294773 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:42.294739 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:37:47.725868 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:37:47.725780 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:10.371170 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:10.371128 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:10.433544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:10.433505 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:10.636757 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:10.636664 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn"] Jul 31 22:38:10.636923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:10.636895 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" containerID="cri-o://3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c" gracePeriod=30 Jul 31 22:38:10.656992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:10.656961 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:10.901693 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:10.901577 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:12.634750 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:12.634713 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:38:15.892544 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.892520 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:38:15.992230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992201 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-socket\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992239 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-podinfo\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992284 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-token\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992321 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-credential-socket\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992345 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-data\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992498 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnfr4\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-kube-api-access-wnfr4\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992516 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:38:15.992648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992594 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80105f3c-7475-476e-8bd1-4df4751b40b8-istiod-ca-cert\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992641 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-certs\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992817 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992692 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-envoy\") pod \"80105f3c-7475-476e-8bd1-4df4751b40b8\" (UID: \"80105f3c-7475-476e-8bd1-4df4751b40b8\") " Jul 31 22:38:15.992987 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.992971 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:15.993381 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.993123 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:38:15.993381 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.993197 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:38:15.993381 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.993314 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:38:15.993613 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.993419 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80105f3c-7475-476e-8bd1-4df4751b40b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:38:15.994586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.994563 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:38:15.995175 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.995146 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:38:15.995344 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.995324 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:38:15.995404 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:15.995345 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-kube-api-access-wnfr4" (OuterVolumeSpecName: "kube-api-access-wnfr4") pod "80105f3c-7475-476e-8bd1-4df4751b40b8" (UID: "80105f3c-7475-476e-8bd1-4df4751b40b8"). InnerVolumeSpecName "kube-api-access-wnfr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:38:16.094084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094047 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80105f3c-7475-476e-8bd1-4df4751b40b8-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094076 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094085 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094084 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094094 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094101 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094110 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094117 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80105f3c-7475-476e-8bd1-4df4751b40b8-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.094343 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.094125 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnfr4\" (UniqueName: \"kubernetes.io/projected/80105f3c-7475-476e-8bd1-4df4751b40b8-kube-api-access-wnfr4\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:38:16.444783 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.444736 2554 generic.go:356] "Generic (PLEG): container finished" podID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerID="3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c" exitCode=0 Jul 31 22:38:16.444987 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.444931 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" event={"ID":"80105f3c-7475-476e-8bd1-4df4751b40b8","Type":"ContainerDied","Data":"3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c"} Jul 31 22:38:16.445049 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.445021 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" event={"ID":"80105f3c-7475-476e-8bd1-4df4751b40b8","Type":"ContainerDied","Data":"2982856187c15b7fb894973333c770defb5af53aa3fea4ca841a06b5378f8481"} Jul 31 22:38:16.445098 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.445061 2554 scope.go:117] "RemoveContainer" containerID="3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c" Jul 31 22:38:16.445451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.445271 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn" Jul 31 22:38:16.456004 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.455986 2554 scope.go:117] "RemoveContainer" containerID="3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b" Jul 31 22:38:16.464003 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.463984 2554 scope.go:117] "RemoveContainer" containerID="3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c" Jul 31 22:38:16.464254 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:38:16.464235 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c\": container with ID starting with 3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c not found: ID does not exist" containerID="3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c" Jul 31 22:38:16.464335 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.464267 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c"} err="failed to get container status \"3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c\": rpc error: code = NotFound desc = could not find container \"3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c\": container with ID starting with 3bc58f195b5d18512403fb82a4b8637aa65adaf770618273d6b8dcffb848e67c not found: ID does not exist" Jul 31 22:38:16.464335 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.464291 2554 scope.go:117] "RemoveContainer" containerID="3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b" Jul 31 22:38:16.464526 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:38:16.464508 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b\": container with ID starting with 3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b not found: ID does not exist" containerID="3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b" Jul 31 22:38:16.464567 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.464532 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b"} err="failed to get container status \"3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b\": rpc error: code = NotFound desc = could not find container \"3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b\": container with ID starting with 3b45aeddae31736ef37d205c40ece645e1324dae7b421d996a476fb2dae4509b not found: ID does not exist" Jul 31 22:38:16.480006 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.479981 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn"] Jul 31 22:38:16.484956 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:16.484931 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9c96e5e7-openshift-default-67bc77bb7b-ldrdn"] Jul 31 22:38:17.957436 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:17.957393 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" path="/var/lib/kubelet/pods/80105f3c-7475-476e-8bd1-4df4751b40b8/volumes" Jul 31 22:38:22.295241 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:22.295207 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:22.394111 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:22.394033 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:22.494797 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:22.494762 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:23.154211 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:23.154174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:23.252999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:23.252957 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:23.352829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:23.352789 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:24.802330 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:24.802294 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:25.043806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:25.043770 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:38:31.309036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:31.308952 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:31.368711 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:31.368677 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:31.643435 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:31.643350 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:38:32.010892 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:32.010848 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:38.064232 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:38.064192 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:38.126880 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:38.126846 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:38.207015 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:38.206984 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:38.305085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:38.305051 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:40.140614 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:40.140568 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:38:48.821765 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:48.821730 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:38:49.070053 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:38:49.070014 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:10.093249 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:10.093208 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:16.338580 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:16.338546 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:16.427638 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:16.427588 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:16.731221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:16.731187 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:17.031530 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031446 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t"] Jul 31 22:39:17.031895 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031881 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" Jul 31 22:39:17.031943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031896 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" Jul 31 22:39:17.031943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031921 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="434b4176-d5fc-406f-b39a-bf39ee11d67d" containerName="main" Jul 31 22:39:17.031943 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031927 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="434b4176-d5fc-406f-b39a-bf39ee11d67d" containerName="main" Jul 31 22:39:17.032036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031984 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="434b4176-d5fc-406f-b39a-bf39ee11d67d" containerName="main" Jul 31 22:39:17.032036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031992 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" Jul 31 22:39:17.032036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.031999 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" Jul 31 22:39:17.035017 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.035001 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.037536 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.037516 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-627bf7-openshift-default-dockercfg-qpz54\"" Jul 31 22:39:17.052026 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.051996 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t"] Jul 31 22:39:17.065845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.065807 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:17.118003 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.117963 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-data\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118017 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istiod-ca-cert\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118054 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-socket\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118080 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-podinfo\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118187 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118181 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-certs\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118358 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118208 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwgtf\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-kube-api-access-vwgtf\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118358 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118239 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-token\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118424 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118377 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-credential-socket\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.118472 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.118422 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-envoy\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.219592 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219552 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-data\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.219800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219627 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istiod-ca-cert\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.219800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219658 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-socket\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.219800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219674 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-podinfo\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.219800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219747 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-certs\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219946 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwgtf\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-kube-api-access-vwgtf\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.219990 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-token\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220025 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-data\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220060 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-certs\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220271 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220064 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-credential-socket\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220271 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220109 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-socket\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220271 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220116 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-envoy\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220399 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220336 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-credential-socket\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.220474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.220455 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istiod-ca-cert\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.222132 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.222109 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-envoy\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.222238 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.222192 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-podinfo\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.227748 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.227725 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-token\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.228548 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.228524 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwgtf\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-kube-api-access-vwgtf\") pod \"e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.283525 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.283441 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:17.346464 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.346430 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:17.491655 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.491628 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t"] Jul 31 22:39:17.493594 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:39:17.493564 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36bb9e9f_b5ac_4e7a_b819_f813b6429a16.slice/crio-cb27f7a7dfcb7c8f8c4fcd71c523aece536d425bb65229f79673e61d1e20b691 WatchSource:0}: Error finding container cb27f7a7dfcb7c8f8c4fcd71c523aece536d425bb65229f79673e61d1e20b691: Status 404 returned error can't find the container with id cb27f7a7dfcb7c8f8c4fcd71c523aece536d425bb65229f79673e61d1e20b691 Jul 31 22:39:17.495806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.495772 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:17.495912 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.495853 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:17.495912 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.495899 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:17.748851 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.748809 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" event={"ID":"36bb9e9f-b5ac-4e7a-b819-f813b6429a16","Type":"ContainerStarted","Data":"25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47"} Jul 31 22:39:17.749036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.748882 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" event={"ID":"36bb9e9f-b5ac-4e7a-b819-f813b6429a16","Type":"ContainerStarted","Data":"cb27f7a7dfcb7c8f8c4fcd71c523aece536d425bb65229f79673e61d1e20b691"} Jul 31 22:39:17.769036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:17.768973 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podStartSLOduration=0.768952751 podStartE2EDuration="768.952751ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:39:17.768822412 +0000 UTC m=+2382.333670993" watchObservedRunningTime="2026-07-31 22:39:17.768952751 +0000 UTC m=+2382.333801337" Jul 31 22:39:18.347454 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:18.347411 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:18.352529 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:18.352503 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:18.753452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:18.753418 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:18.754502 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:18.754479 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:19.743036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:19.742996 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:19.839838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:19.839800 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:20.874671 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:20.874633 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:21.093042 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:21.092999 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:21.386828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:21.386792 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:21.486300 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:21.486265 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:23.545433 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:23.545400 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:39:33.755751 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.755710 2554 patch_prober.go:28] interesting pod/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 31 22:39:33.756174 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.755785 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 31 22:39:33.844216 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.844182 2554 generic.go:356] "Generic (PLEG): container finished" podID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerID="25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47" exitCode=0 Jul 31 22:39:33.844380 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.844244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" event={"ID":"36bb9e9f-b5ac-4e7a-b819-f813b6429a16","Type":"ContainerDied","Data":"25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47"} Jul 31 22:39:33.844667 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.844653 2554 scope.go:117] "RemoveContainer" containerID="25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47" Jul 31 22:39:33.845716 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.845683 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:33.845787 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.845770 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:33.845833 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:33.845819 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:34.851366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:34.851333 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" event={"ID":"36bb9e9f-b5ac-4e7a-b819-f813b6429a16","Type":"ContainerStarted","Data":"68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71"} Jul 31 22:39:35.347141 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:35.347106 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:35.347368 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:35.347303 2554 patch_prober.go:28] interesting pod/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 31 22:39:35.347442 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:35.347357 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 31 22:39:36.117919 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:36.117890 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:39:36.121217 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:36.121195 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:39:36.347000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:36.346963 2554 patch_prober.go:28] interesting pod/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 31 22:39:36.347186 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:36.347026 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 31 22:39:37.347752 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:37.347714 2554 patch_prober.go:28] interesting pod/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 31 22:39:37.348152 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:37.347772 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 31 22:39:38.347532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:38.347496 2554 patch_prober.go:28] interesting pod/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 31 22:39:38.347748 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:38.347563 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 31 22:39:39.347821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:39.347778 2554 patch_prober.go:28] interesting pod/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 31 22:39:39.348216 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:39.347837 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 31 22:39:40.350791 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:40.350759 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:40.351214 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:40.351130 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:40.351953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:40.351932 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:39:44.850809 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:44.850773 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:45.208087 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.208053 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:45.601983 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.601907 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf"] Jul 31 22:39:45.602303 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.602291 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" Jul 31 22:39:45.602351 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.602305 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="80105f3c-7475-476e-8bd1-4df4751b40b8" containerName="istio-proxy" Jul 31 22:39:45.605355 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.605337 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.608764 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.608745 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ae5bab-openshift-default-dockercfg-59d4r\"" Jul 31 22:39:45.635909 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.635876 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf"] Jul 31 22:39:45.657000 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.656970 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:45.698658 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698622 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-envoy\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.698846 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698675 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-data\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.698846 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698828 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24c93df1-4172-449d-ad89-b449f3521309-istio-podinfo\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.698970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-credential-socket\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.698970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698887 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crj9c\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-kube-api-access-crj9c\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.698970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698922 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24c93df1-4172-449d-ad89-b449f3521309-istiod-ca-cert\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.698970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698949 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-certs\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.699144 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.698974 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-istio-token\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.699144 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.699007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-socket\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800138 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800102 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24c93df1-4172-449d-ad89-b449f3521309-istio-podinfo\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800138 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800142 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-credential-socket\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800160 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crj9c\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-kube-api-access-crj9c\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800185 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24c93df1-4172-449d-ad89-b449f3521309-istiod-ca-cert\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800213 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-certs\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800242 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-istio-token\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800275 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-socket\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800333 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-envoy\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800388 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800361 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-data\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800815 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800626 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-credential-socket\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800815 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800650 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-certs\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800940 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800915 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-data\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.800998 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.800938 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-socket\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.801279 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.801256 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24c93df1-4172-449d-ad89-b449f3521309-istiod-ca-cert\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.802898 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.802881 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-envoy\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.803041 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.803023 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24c93df1-4172-449d-ad89-b449f3521309-istio-podinfo\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.811871 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.811843 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-istio-token\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.812183 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.812162 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crj9c\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-kube-api-access-crj9c\") pod \"e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:45.916217 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:45.916132 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:46.088526 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.088491 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf"] Jul 31 22:39:46.091888 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:39:46.091861 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c93df1_4172_449d_ad89_b449f3521309.slice/crio-8014b95d3a88e82d3044280e549dc9193612cbc74d423fd2ec43a7fa79365b85 WatchSource:0}: Error finding container 8014b95d3a88e82d3044280e549dc9193612cbc74d423fd2ec43a7fa79365b85: Status 404 returned error can't find the container with id 8014b95d3a88e82d3044280e549dc9193612cbc74d423fd2ec43a7fa79365b85 Jul 31 22:39:46.094611 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.094568 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:46.094703 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.094653 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:46.094703 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.094687 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:39:46.486802 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.486767 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:46.913018 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.912930 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" event={"ID":"24c93df1-4172-449d-ad89-b449f3521309","Type":"ContainerStarted","Data":"9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe"} Jul 31 22:39:46.913018 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.913012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" event={"ID":"24c93df1-4172-449d-ad89-b449f3521309","Type":"ContainerStarted","Data":"8014b95d3a88e82d3044280e549dc9193612cbc74d423fd2ec43a7fa79365b85"} Jul 31 22:39:46.916315 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.916297 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:46.945976 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:46.945926 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podStartSLOduration=1.945910051 podStartE2EDuration="1.945910051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:39:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:39:46.937714738 +0000 UTC m=+2411.502563321" watchObservedRunningTime="2026-07-31 22:39:46.945910051 +0000 UTC m=+2411.510758632" Jul 31 22:39:47.921518 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:47.921480 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:48.925669 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:48.925638 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:48.926594 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:48.926572 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:39:49.110762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:49.110723 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:49.211655 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:49.211620 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:50.332932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:50.332898 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:50.439320 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:50.439281 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:51.037985 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:51.037947 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:51.129521 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:51.129485 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:39:53.764866 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:39:53.764829 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:40:03.927325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:03.927230 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 31 22:40:03.927325 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:03.927289 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 31 22:40:04.002317 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:04.002284 2554 generic.go:356] "Generic (PLEG): container finished" podID="24c93df1-4172-449d-ad89-b449f3521309" containerID="9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe" exitCode=0 Jul 31 22:40:04.002489 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:04.002357 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" event={"ID":"24c93df1-4172-449d-ad89-b449f3521309","Type":"ContainerDied","Data":"9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe"} Jul 31 22:40:04.002756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:04.002743 2554 scope.go:117] "RemoveContainer" containerID="9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe" Jul 31 22:40:04.003837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:04.003809 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:40:04.003901 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:04.003879 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:40:04.003941 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:04.003907 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:40:05.010123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:05.010085 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" event={"ID":"24c93df1-4172-449d-ad89-b449f3521309","Type":"ContainerStarted","Data":"576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194"} Jul 31 22:40:05.917198 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:05.917162 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:40:05.917384 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:05.917361 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 31 22:40:05.917433 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:05.917416 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 31 22:40:06.916480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:06.916442 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 31 22:40:06.916911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:06.916501 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 31 22:40:07.917202 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:07.917162 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 31 22:40:07.917629 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:07.917220 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 31 22:40:08.917442 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:08.917405 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 31 22:40:08.917875 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:08.917463 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 31 22:40:09.916837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:09.916798 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 31 22:40:09.917009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:09.916862 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 31 22:40:10.920829 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:10.920799 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:40:10.921208 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:10.921073 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:40:10.921722 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:10.921706 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:40:14.077475 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:14.077441 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:40:27.365029 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:27.364997 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:40:29.838753 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:29.838719 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:40:37.338562 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:37.338523 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:40:54.465662 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:54.465625 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:40:57.351774 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:40:57.351731 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:41:00.236503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:00.236464 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:06.440364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:06.440330 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:41:38.971631 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:38.971530 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:39.899453 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:39.899417 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:42.342671 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:42.342627 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:41:50.475757 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:50.475705 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:50.576009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:50.575977 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:50.661489 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:50.661453 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:50.718725 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:50.718686 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:51.057784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:51.057748 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:52.843131 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:52.843092 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:41:56.769653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:56.769612 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:56.866992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:56.866956 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:56.967335 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:56.967297 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:57.161362 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:57.161279 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t"] Jul 31 22:41:57.161578 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:57.161548 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" containerID="cri-o://68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71" gracePeriod=30 Jul 31 22:41:57.182927 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:57.182888 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:41:57.414786 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:41:57.414681 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:01.563071 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:01.563018 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:02.426632 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.426554 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:42:02.479569 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479531 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-socket\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.479792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479641 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-credential-socket\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.479792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479678 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-data\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.479792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479714 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-podinfo\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.479792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479747 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-token\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.480010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479808 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-envoy\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.480010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479832 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istiod-ca-cert\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.480010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479858 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwgtf\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-kube-api-access-vwgtf\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.480010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479864 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:02.480010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479887 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-certs\") pod \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\" (UID: \"36bb9e9f-b5ac-4e7a-b819-f813b6429a16\") " Jul 31 22:42:02.480010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.479975 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:02.480298 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.480206 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.480298 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.480236 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.480298 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.480245 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:42:02.480803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.480694 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-data" (OuterVolumeSpecName: "istio-data") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:02.480937 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.480882 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:02.482594 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.482572 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-token" (OuterVolumeSpecName: "istio-token") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:42:02.482594 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.482625 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-kube-api-access-vwgtf" (OuterVolumeSpecName: "kube-api-access-vwgtf") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "kube-api-access-vwgtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:42:02.482932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.482907 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:02.483219 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.483194 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36bb9e9f-b5ac-4e7a-b819-f813b6429a16" (UID: "36bb9e9f-b5ac-4e7a-b819-f813b6429a16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:42:02.581157 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581118 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.581157 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581149 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.581157 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581158 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.581157 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581165 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwgtf\" (UniqueName: \"kubernetes.io/projected/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-kube-api-access-vwgtf\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.581845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581174 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.581845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581183 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.581845 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.581192 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36bb9e9f-b5ac-4e7a-b819-f813b6429a16-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:02.608065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.608027 2554 generic.go:356] "Generic (PLEG): container finished" podID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerID="68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71" exitCode=0 Jul 31 22:42:02.608231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.608076 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" Jul 31 22:42:02.608231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.608109 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" event={"ID":"36bb9e9f-b5ac-4e7a-b819-f813b6429a16","Type":"ContainerDied","Data":"68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71"} Jul 31 22:42:02.608231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.608167 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t" event={"ID":"36bb9e9f-b5ac-4e7a-b819-f813b6429a16","Type":"ContainerDied","Data":"cb27f7a7dfcb7c8f8c4fcd71c523aece536d425bb65229f79673e61d1e20b691"} Jul 31 22:42:02.608231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.608198 2554 scope.go:117] "RemoveContainer" containerID="68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71" Jul 31 22:42:02.617473 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.617454 2554 scope.go:117] "RemoveContainer" containerID="25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47" Jul 31 22:42:02.625656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.625638 2554 scope.go:117] "RemoveContainer" containerID="68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71" Jul 31 22:42:02.625909 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:42:02.625890 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71\": container with ID starting with 68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71 not found: ID does not exist" containerID="68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71" Jul 31 22:42:02.625955 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.625917 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71"} err="failed to get container status \"68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71\": rpc error: code = NotFound desc = could not find container \"68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71\": container with ID starting with 68ba74d5bb58f8d2af6266144bdcc86917ef5861615da675d83deb1119dcab71 not found: ID does not exist" Jul 31 22:42:02.625955 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.625935 2554 scope.go:117] "RemoveContainer" containerID="25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47" Jul 31 22:42:02.626175 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:42:02.626156 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47\": container with ID starting with 25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47 not found: ID does not exist" containerID="25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47" Jul 31 22:42:02.626234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.626180 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47"} err="failed to get container status \"25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47\": rpc error: code = NotFound desc = could not find container \"25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47\": container with ID starting with 25f18d1456c7564e9957fa68c7f05aaaa63b68e87acddd65b10b43e75f1c3d47 not found: ID does not exist" Jul 31 22:42:02.629878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.629859 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t"] Jul 31 22:42:02.639378 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.639355 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-627bf7-openshift-default-579c997c78-k2d8t"] Jul 31 22:42:02.938878 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:02.938839 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:03.733691 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:03.733652 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:42:03.957302 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:03.957267 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" path="/var/lib/kubelet/pods/36bb9e9f-b5ac-4e7a-b819-f813b6429a16/volumes" Jul 31 22:42:05.928683 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:05.928645 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:08.116327 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.116294 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:08.185720 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.185678 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:08.350861 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.350828 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf"] Jul 31 22:42:08.351096 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.351064 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" containerID="cri-o://576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194" gracePeriod=30 Jul 31 22:42:08.374733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.374636 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:08.610082 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.610046 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:08.961056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:08.961024 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:09.266111 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.266030 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc"] Jul 31 22:42:09.266622 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.266587 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" Jul 31 22:42:09.266713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.266625 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" Jul 31 22:42:09.266713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.266654 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" Jul 31 22:42:09.266713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.266663 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" Jul 31 22:42:09.266858 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.266758 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" Jul 31 22:42:09.271518 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.271497 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.274314 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.274295 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f117f4-openshift-default-dockercfg-6mxs4\"" Jul 31 22:42:09.282563 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.282543 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc"] Jul 31 22:42:09.291184 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.291159 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:09.341669 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341642 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-certs\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.341834 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341681 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-credential-socket\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.341834 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341718 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-token\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.341834 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341757 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-podinfo\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.341834 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341806 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc00b8f7-ef10-4366-9809-3803c0c295bf-istiod-ca-cert\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.342023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341835 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-data\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.342023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-socket\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.342023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341918 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbx7l\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-kube-api-access-pbx7l\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.342023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.341989 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-envoy\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.442821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.442775 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-certs\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.442821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.442817 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-credential-socket\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.442840 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-token\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.442869 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-podinfo\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.442905 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc00b8f7-ef10-4366-9809-3803c0c295bf-istiod-ca-cert\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443115 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.442936 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-data\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443322 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443204 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-socket\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443322 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443269 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbx7l\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-kube-api-access-pbx7l\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443322 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443311 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-envoy\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443473 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443334 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-data\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443473 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443311 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-credential-socket\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443524 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-socket\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443658 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443628 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-certs\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.443766 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.443742 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc00b8f7-ef10-4366-9809-3803c0c295bf-istiod-ca-cert\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.445398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.445373 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-envoy\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.445611 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.445582 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-podinfo\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.450799 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.450771 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-token\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.451299 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.451278 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbx7l\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-kube-api-access-pbx7l\") pod \"e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.559036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.558949 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:09.584453 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.584420 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:09.742748 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.742720 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc"] Jul 31 22:42:09.744882 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:42:09.744851 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc00b8f7_ef10_4366_9809_3803c0c295bf.slice/crio-0f653b289d5e7eb5c2711d29e0dc2cce5356aa99148089c13ba826b0df361bd3 WatchSource:0}: Error finding container 0f653b289d5e7eb5c2711d29e0dc2cce5356aa99148089c13ba826b0df361bd3: Status 404 returned error can't find the container with id 0f653b289d5e7eb5c2711d29e0dc2cce5356aa99148089c13ba826b0df361bd3 Jul 31 22:42:09.747181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.747142 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:09.747313 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.747223 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:09.747313 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:09.747268 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:10.662415 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:10.662380 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" event={"ID":"dc00b8f7-ef10-4366-9809-3803c0c295bf","Type":"ContainerStarted","Data":"530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4"} Jul 31 22:42:10.662837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:10.662439 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" event={"ID":"dc00b8f7-ef10-4366-9809-3803c0c295bf","Type":"ContainerStarted","Data":"0f653b289d5e7eb5c2711d29e0dc2cce5356aa99148089c13ba826b0df361bd3"} Jul 31 22:42:10.686234 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:10.686180 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podStartSLOduration=1.686164622 podStartE2EDuration="1.686164622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:42:10.684085934 +0000 UTC m=+2555.248934515" watchObservedRunningTime="2026-07-31 22:42:10.686164622 +0000 UTC m=+2555.251013203" Jul 31 22:42:10.923809 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:10.923719 2554 patch_prober.go:28] interesting pod/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:42:10.923809 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:10.923783 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:42:11.584592 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:11.584556 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:11.589749 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:11.589716 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:11.668255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:11.668221 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:11.669306 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:11.669285 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:12.610214 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:12.610174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:12.709936 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:12.709906 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:13.600379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.600354 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:42:13.679420 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.679330 2554 generic.go:356] "Generic (PLEG): container finished" podID="24c93df1-4172-449d-ad89-b449f3521309" containerID="576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194" exitCode=0 Jul 31 22:42:13.679420 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.679385 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" Jul 31 22:42:13.679420 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.679405 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" event={"ID":"24c93df1-4172-449d-ad89-b449f3521309","Type":"ContainerDied","Data":"576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194"} Jul 31 22:42:13.679736 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.679464 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf" event={"ID":"24c93df1-4172-449d-ad89-b449f3521309","Type":"ContainerDied","Data":"8014b95d3a88e82d3044280e549dc9193612cbc74d423fd2ec43a7fa79365b85"} Jul 31 22:42:13.679736 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.679494 2554 scope.go:117] "RemoveContainer" containerID="576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194" Jul 31 22:42:13.684922 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.684900 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-certs\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685032 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.684954 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-istio-token\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685032 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.684991 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-envoy\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685149 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685057 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crj9c\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-kube-api-access-crj9c\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685149 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685091 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-socket\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685149 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685121 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24c93df1-4172-449d-ad89-b449f3521309-istio-podinfo\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685163 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-credential-socket\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685190 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:13.685348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685197 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24c93df1-4172-449d-ad89-b449f3521309-istiod-ca-cert\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685348 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685240 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-data\") pod \"24c93df1-4172-449d-ad89-b449f3521309\" (UID: \"24c93df1-4172-449d-ad89-b449f3521309\") " Jul 31 22:42:13.685620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685398 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:13.685620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685559 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.685620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685580 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.685620 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685581 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:13.685852 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685804 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-data" (OuterVolumeSpecName: "istio-data") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:13.686773 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.685933 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c93df1-4172-449d-ad89-b449f3521309-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:42:13.688256 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.688219 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-istio-token" (OuterVolumeSpecName: "istio-token") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:42:13.688366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.688276 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:42:13.688366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.688294 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-kube-api-access-crj9c" (OuterVolumeSpecName: "kube-api-access-crj9c") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "kube-api-access-crj9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:42:13.688366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.688290 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24c93df1-4172-449d-ad89-b449f3521309-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24c93df1-4172-449d-ad89-b449f3521309" (UID: "24c93df1-4172-449d-ad89-b449f3521309"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:42:13.690036 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.690015 2554 scope.go:117] "RemoveContainer" containerID="9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe" Jul 31 22:42:13.711886 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.711855 2554 scope.go:117] "RemoveContainer" containerID="576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194" Jul 31 22:42:13.712288 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:42:13.712256 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194\": container with ID starting with 576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194 not found: ID does not exist" containerID="576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194" Jul 31 22:42:13.712346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.712303 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194"} err="failed to get container status \"576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194\": rpc error: code = NotFound desc = could not find container \"576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194\": container with ID starting with 576ad6352c8b92d06708c6c1314891799e532aafc56d7333c9568e55e2c1e194 not found: ID does not exist" Jul 31 22:42:13.712346 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.712329 2554 scope.go:117] "RemoveContainer" containerID="9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe" Jul 31 22:42:13.712697 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:42:13.712672 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe\": container with ID starting with 9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe not found: ID does not exist" containerID="9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe" Jul 31 22:42:13.712793 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.712707 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe"} err="failed to get container status \"9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe\": rpc error: code = NotFound desc = could not find container \"9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe\": container with ID starting with 9243527f19e0901e04e6454dc9d1d06d443371ace8b35be2981cb2f87f7e3dbe not found: ID does not exist" Jul 31 22:42:13.747524 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.747418 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786293 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24c93df1-4172-449d-ad89-b449f3521309-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786324 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786338 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786353 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786367 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crj9c\" (UniqueName: \"kubernetes.io/projected/24c93df1-4172-449d-ad89-b449f3521309-kube-api-access-crj9c\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786382 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24c93df1-4172-449d-ad89-b449f3521309-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.786509 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.786397 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24c93df1-4172-449d-ad89-b449f3521309-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:42:13.997281 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:13.997248 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf"] Jul 31 22:42:14.001455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:14.001430 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ae5bab-openshift-default-86cf7cd74d-7wzzf"] Jul 31 22:42:14.174883 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:14.174846 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:14.282224 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:14.282146 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:14.361364 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:14.361316 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:15.957211 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:15.957172 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c93df1-4172-449d-ad89-b449f3521309" path="/var/lib/kubelet/pods/24c93df1-4172-449d-ad89-b449f3521309/volumes" Jul 31 22:42:16.239865 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:16.239769 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:42:26.670177 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.670144 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 31 22:42:26.670577 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.670200 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 31 22:42:26.753539 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.753508 2554 generic.go:356] "Generic (PLEG): container finished" podID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerID="530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4" exitCode=0 Jul 31 22:42:26.753701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.753585 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" event={"ID":"dc00b8f7-ef10-4366-9809-3803c0c295bf","Type":"ContainerDied","Data":"530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4"} Jul 31 22:42:26.754012 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.753995 2554 scope.go:117] "RemoveContainer" containerID="530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4" Jul 31 22:42:26.754923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.754897 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:26.754984 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.754968 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:26.755023 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:26.755005 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:27.761175 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:27.761138 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" event={"ID":"dc00b8f7-ef10-4366-9809-3803c0c295bf","Type":"ContainerStarted","Data":"ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38"} Jul 31 22:42:28.585173 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:28.585126 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:28.585497 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:28.585186 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 31 22:42:28.585723 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:28.585685 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 31 22:42:29.585212 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:29.585164 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 31 22:42:29.585733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:29.585255 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 31 22:42:30.586413 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:30.586363 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 31 22:42:30.586924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:30.586447 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 31 22:42:31.585442 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:31.585403 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 31 22:42:31.585788 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:31.585754 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 31 22:42:32.585278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:32.585237 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 31 22:42:32.585902 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:32.585870 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 31 22:42:33.589406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:33.589373 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:33.589825 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:33.589652 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:33.590342 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:33.590326 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:42:37.645576 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:37.645534 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:38.134850 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.134812 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:38.439566 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.439527 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd"] Jul 31 22:42:38.440025 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440006 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" Jul 31 22:42:38.440123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440027 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" Jul 31 22:42:38.440123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440041 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" Jul 31 22:42:38.440123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440050 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" Jul 31 22:42:38.440278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440148 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="36bb9e9f-b5ac-4e7a-b819-f813b6429a16" containerName="istio-proxy" Jul 31 22:42:38.440278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440163 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" Jul 31 22:42:38.440278 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.440177 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c93df1-4172-449d-ad89-b449f3521309" containerName="istio-proxy" Jul 31 22:42:38.442885 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.442865 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.445841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.445821 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-84098f-openshift-default-dockercfg-lth5v\"" Jul 31 22:42:38.457295 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.457271 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd"] Jul 31 22:42:38.473268 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.473233 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:38.495517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495486 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-certs\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495521 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-envoy\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495543 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-token\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495680 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-socket\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495710 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-podinfo\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495728 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25znp\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-kube-api-access-25znp\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495786 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615e3dd4-8283-414d-a199-aa4b4df5bd58-istiod-ca-cert\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495827 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-data\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.495938 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.495859 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-credential-socket\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.572757 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.572721 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:38.597179 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597146 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-socket\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597179 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597179 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-podinfo\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597199 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25znp\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-kube-api-access-25znp\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597228 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615e3dd4-8283-414d-a199-aa4b4df5bd58-istiod-ca-cert\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597251 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-data\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597276 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-credential-socket\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597406 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597337 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-certs\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597674 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597412 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-envoy\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597674 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597448 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-token\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-credential-socket\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.597911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.597887 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-data\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.598054 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.598033 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615e3dd4-8283-414d-a199-aa4b4df5bd58-istiod-ca-cert\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.598110 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.598087 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-socket\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.598161 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.598122 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-certs\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.599641 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.599615 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-envoy\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.599920 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.599899 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-podinfo\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.604889 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.604862 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-token\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.605085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.605068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25znp\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-kube-api-access-25znp\") pod \"e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.754215 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.754129 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:38.892365 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.892338 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd"] Jul 31 22:42:38.894394 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:42:38.894365 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615e3dd4_8283_414d_a199_aa4b4df5bd58.slice/crio-c6c33af4cb4c4625d4749934a04d8fb10e9c2efbb329f9f1a807cdae5b5b1459 WatchSource:0}: Error finding container c6c33af4cb4c4625d4749934a04d8fb10e9c2efbb329f9f1a807cdae5b5b1459: Status 404 returned error can't find the container with id c6c33af4cb4c4625d4749934a04d8fb10e9c2efbb329f9f1a807cdae5b5b1459 Jul 31 22:42:38.897252 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.897210 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:38.897347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.897299 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:38.897347 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:38.897340 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:39.829470 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:39.829431 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" event={"ID":"615e3dd4-8283-414d-a199-aa4b4df5bd58","Type":"ContainerStarted","Data":"c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1"} Jul 31 22:42:39.829899 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:39.829487 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" event={"ID":"615e3dd4-8283-414d-a199-aa4b4df5bd58","Type":"ContainerStarted","Data":"c6c33af4cb4c4625d4749934a04d8fb10e9c2efbb329f9f1a807cdae5b5b1459"} Jul 31 22:42:39.849214 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:39.849174 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podStartSLOduration=1.84915914 podStartE2EDuration="1.84915914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:42:39.84795365 +0000 UTC m=+2584.412802234" watchObservedRunningTime="2026-07-31 22:42:39.84915914 +0000 UTC m=+2584.414007722" Jul 31 22:42:40.755104 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:40.755073 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:40.759526 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:40.759501 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:40.834142 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:40.834104 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:40.835127 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:40.835108 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:41.780869 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:41.780835 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:41.874945 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:41.874899 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:42.904136 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:42.904102 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:43.038198 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:43.038158 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:43.654858 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:43.654820 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:43.756967 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:43.756932 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:42:46.336063 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:46.336029 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:42:56.224443 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.223159 2554 patch_prober.go:28] interesting pod/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:60762->10.132.0.55:15021: read: connection reset by peer" start-of-body= Jul 31 22:42:56.224443 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.223224 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:60762->10.132.0.55:15021: read: connection reset by peer" Jul 31 22:42:56.923195 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.923162 2554 generic.go:356] "Generic (PLEG): container finished" podID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerID="c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1" exitCode=0 Jul 31 22:42:56.923443 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.923232 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" event={"ID":"615e3dd4-8283-414d-a199-aa4b4df5bd58","Type":"ContainerDied","Data":"c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1"} Jul 31 22:42:56.923696 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.923677 2554 scope.go:117] "RemoveContainer" containerID="c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1" Jul 31 22:42:56.924778 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.924747 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:56.924852 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.924816 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:56.924852 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:56.924848 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:42:57.929937 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:57.929908 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" event={"ID":"615e3dd4-8283-414d-a199-aa4b4df5bd58","Type":"ContainerStarted","Data":"194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1"} Jul 31 22:42:58.754727 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:58.754685 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:42:58.754968 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:58.754943 2554 patch_prober.go:28] interesting pod/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 31 22:42:58.755056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:58.755004 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 31 22:42:59.754948 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:59.754914 2554 patch_prober.go:28] interesting pod/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 31 22:42:59.755358 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:42:59.754970 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 31 22:43:00.755244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:00.755203 2554 patch_prober.go:28] interesting pod/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 31 22:43:00.755762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:00.755269 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 31 22:43:01.754684 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:01.754648 2554 patch_prober.go:28] interesting pod/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 31 22:43:01.754859 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:01.754701 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 31 22:43:02.755169 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:02.755133 2554 patch_prober.go:28] interesting pod/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 31 22:43:02.755562 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:02.755188 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 31 22:43:03.758258 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:03.758220 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:43:03.758733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:03.758512 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:43:03.759249 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:03.759228 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:43:06.825434 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:06.825402 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:25.378717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:25.378671 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:25.442321 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:25.442277 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:25.494754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:25.494714 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:25.587369 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:25.587332 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:25.972243 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:25.972208 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:27.838185 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:27.838150 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:43:31.864114 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:31.864081 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:31.964833 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:31.964794 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:32.128868 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:32.128785 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc"] Jul 31 22:43:32.129087 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:32.129052 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" containerID="cri-o://ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38" gracePeriod=30 Jul 31 22:43:32.144269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:32.144242 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:32.392710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:32.392591 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:33.592546 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:33.592506 2554 patch_prober.go:28] interesting pod/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:43:33.592929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:33.592568 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:43:37.387888 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.387864 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:43:37.456993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.456963 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-certs\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.456993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.456995 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc00b8f7-ef10-4366-9809-3803c0c295bf-istiod-ca-cert\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457240 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457054 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-data\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457240 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457119 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-envoy\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457365 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457301 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbx7l\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-kube-api-access-pbx7l\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457365 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457351 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-credential-socket\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457475 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457433 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-token\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457530 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457503 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-podinfo\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457587 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457547 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-socket\") pod \"dc00b8f7-ef10-4366-9809-3803c0c295bf\" (UID: \"dc00b8f7-ef10-4366-9809-3803c0c295bf\") " Jul 31 22:43:37.457587 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457556 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:37.457747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457575 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:37.457975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457829 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc00b8f7-ef10-4366-9809-3803c0c295bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:43:37.457975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457821 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:37.457975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457890 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.457975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457911 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.457975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.457922 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:37.459747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.459713 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:43:37.459864 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.459781 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:43:37.459864 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.459807 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-kube-api-access-pbx7l" (OuterVolumeSpecName: "kube-api-access-pbx7l") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "kube-api-access-pbx7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:43:37.459864 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.459822 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc00b8f7-ef10-4366-9809-3803c0c295bf" (UID: "dc00b8f7-ef10-4366-9809-3803c0c295bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:37.558733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558702 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc00b8f7-ef10-4366-9809-3803c0c295bf-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.558733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558727 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.558733 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558737 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.558966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558746 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbx7l\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-kube-api-access-pbx7l\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.558966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558754 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.558966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558762 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc00b8f7-ef10-4366-9809-3803c0c295bf-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:37.558966 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:37.558769 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc00b8f7-ef10-4366-9809-3803c0c295bf-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:38.131656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.131591 2554 generic.go:356] "Generic (PLEG): container finished" podID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerID="ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38" exitCode=0 Jul 31 22:43:38.131656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.131638 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" event={"ID":"dc00b8f7-ef10-4366-9809-3803c0c295bf","Type":"ContainerDied","Data":"ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38"} Jul 31 22:43:38.131884 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.131673 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" Jul 31 22:43:38.131884 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.131702 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc" event={"ID":"dc00b8f7-ef10-4366-9809-3803c0c295bf","Type":"ContainerDied","Data":"0f653b289d5e7eb5c2711d29e0dc2cce5356aa99148089c13ba826b0df361bd3"} Jul 31 22:43:38.131884 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.131727 2554 scope.go:117] "RemoveContainer" containerID="ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38" Jul 31 22:43:38.140792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.140771 2554 scope.go:117] "RemoveContainer" containerID="530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4" Jul 31 22:43:38.148831 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.148791 2554 scope.go:117] "RemoveContainer" containerID="ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38" Jul 31 22:43:38.149156 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:43:38.149133 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38\": container with ID starting with ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38 not found: ID does not exist" containerID="ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38" Jul 31 22:43:38.149254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.149162 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38"} err="failed to get container status \"ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38\": rpc error: code = NotFound desc = could not find container \"ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38\": container with ID starting with ebe353e0a4cbac27285d676c5a13922ead6c22f13ba0a28dc998c3b890bfdf38 not found: ID does not exist" Jul 31 22:43:38.149254 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.149179 2554 scope.go:117] "RemoveContainer" containerID="530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4" Jul 31 22:43:38.149540 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:43:38.149522 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4\": container with ID starting with 530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4 not found: ID does not exist" containerID="530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4" Jul 31 22:43:38.149616 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.149547 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4"} err="failed to get container status \"530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4\": rpc error: code = NotFound desc = could not find container \"530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4\": container with ID starting with 530da125801f4187902fb6b344b631c148249ee4712f73bafbbea73d2abdf4f4 not found: ID does not exist" Jul 31 22:43:38.151247 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.151227 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc"] Jul 31 22:43:38.155071 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:38.155050 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f117f4-openshift-default-64bbd4c9b6-l89jc"] Jul 31 22:43:39.957333 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:39.957298 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" path="/var/lib/kubelet/pods/dc00b8f7-ef10-4366-9809-3803c0c295bf/volumes" Jul 31 22:43:48.067574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:48.067540 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:48.147016 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:48.146983 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:48.202398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:48.202362 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:48.260860 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:48.260815 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:48.639944 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:48.639914 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:49.341491 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:49.341455 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:43:50.713156 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:50.713124 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:54.338564 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:54.338529 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:54.438792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:54.438752 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:54.599490 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:54.599401 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd"] Jul 31 22:43:54.599710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:54.599673 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" containerID="cri-o://194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1" gracePeriod=30 Jul 31 22:43:54.623381 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:54.623350 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:54.847152 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:54.847115 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:55.225457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.225407 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:55.528805 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.528730 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd"] Jul 31 22:43:55.529221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.529121 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" Jul 31 22:43:55.529221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.529133 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" Jul 31 22:43:55.529221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.529156 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" Jul 31 22:43:55.529221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.529162 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" Jul 31 22:43:55.529221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.529218 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" Jul 31 22:43:55.529221 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.529226 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc00b8f7-ef10-4366-9809-3803c0c295bf" containerName="istio-proxy" Jul 31 22:43:55.532664 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.532646 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.534997 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.534981 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-08271f-openshift-default-dockercfg-mtl49\"" Jul 31 22:43:55.544106 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.544081 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd"] Jul 31 22:43:55.554530 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.554499 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:55.725515 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725484 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-envoy\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725718 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725522 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-credential-socket\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725718 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725554 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9r6x\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-kube-api-access-j9r6x\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725718 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725650 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-socket\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725718 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725683 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-token\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725741 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-data\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725773 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-certs\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725822 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istiod-ca-cert\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.725960 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.725892 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-podinfo\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.826713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826571 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-envoy\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.826713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826658 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-credential-socket\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.826713 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826696 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9r6x\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-kube-api-access-j9r6x\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826735 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-socket\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826841 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-token\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826926 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-data\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.826971 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-certs\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istiod-ca-cert\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827129 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-credential-socket\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827137 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-podinfo\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827374 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-data\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827136 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-socket\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827397 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-certs\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.827798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.827775 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istiod-ca-cert\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.829043 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.829022 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-envoy\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.829917 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.829895 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-podinfo\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.834784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.834760 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-token\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.834885 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.834848 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9r6x\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-kube-api-access-j9r6x\") pod \"e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.844653 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.844634 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:55.982156 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:55.982117 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:56.008836 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.008812 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd"] Jul 31 22:43:56.010772 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:43:56.010735 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8d2f96_dcc7_4e03_b2fe_081c1592168c.slice/crio-0dcd6e9b387493fdf459056a2c71260fad1109f53a16ffae39a1ee440b896488 WatchSource:0}: Error finding container 0dcd6e9b387493fdf459056a2c71260fad1109f53a16ffae39a1ee440b896488: Status 404 returned error can't find the container with id 0dcd6e9b387493fdf459056a2c71260fad1109f53a16ffae39a1ee440b896488 Jul 31 22:43:56.013005 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.012968 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:43:56.013075 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.013053 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:43:56.013118 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.013105 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:43:56.227052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.227007 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" event={"ID":"4b8d2f96-dcc7-4e03-b2fe-081c1592168c","Type":"ContainerStarted","Data":"0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734"} Jul 31 22:43:56.227226 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.227087 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" event={"ID":"4b8d2f96-dcc7-4e03-b2fe-081c1592168c","Type":"ContainerStarted","Data":"0dcd6e9b387493fdf459056a2c71260fad1109f53a16ffae39a1ee440b896488"} Jul 31 22:43:56.246619 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.246542 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podStartSLOduration=1.246527219 podStartE2EDuration="1.246527219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:43:56.24528994 +0000 UTC m=+2660.810138524" watchObservedRunningTime="2026-07-31 22:43:56.246527219 +0000 UTC m=+2660.811375801" Jul 31 22:43:56.844905 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:56.844873 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:57.849870 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:57.849840 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:58.168710 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:58.168621 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:58.237471 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:58.237444 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:58.238400 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:58.238381 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:43:58.271218 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:58.271185 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:59.301790 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.301760 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:59.433124 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.433091 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:59.822641 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.822593 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:59.854911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.854881 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:43:59.865717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.865660 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615e3dd4-8283-414d-a199-aa4b4df5bd58-istiod-ca-cert\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.865717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.865699 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-podinfo\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.865717 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.865716 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-token\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.865933 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.865750 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25znp\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-kube-api-access-25znp\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.865933 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.865905 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-certs\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.866041 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.865993 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615e3dd4-8283-414d-a199-aa4b4df5bd58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:43:59.866041 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866003 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-data\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.866147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866046 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-credential-socket\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.866147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866078 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-socket\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.866147 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866110 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-envoy\") pod \"615e3dd4-8283-414d-a199-aa4b4df5bd58\" (UID: \"615e3dd4-8283-414d-a199-aa4b4df5bd58\") " Jul 31 22:43:59.866299 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866189 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:59.866299 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866252 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-data" (OuterVolumeSpecName: "istio-data") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:59.866430 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866376 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:59.866501 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866448 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:59.866501 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866491 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615e3dd4-8283-414d-a199-aa4b4df5bd58-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.866586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866512 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.866586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866523 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.866586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.866531 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.867977 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.867952 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:43:59.868288 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.868266 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:43:59.868363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.868344 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-token" (OuterVolumeSpecName: "istio-token") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:43:59.868404 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.868366 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-kube-api-access-25znp" (OuterVolumeSpecName: "kube-api-access-25znp") pod "615e3dd4-8283-414d-a199-aa4b4df5bd58" (UID: "615e3dd4-8283-414d-a199-aa4b4df5bd58"). InnerVolumeSpecName "kube-api-access-25znp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:43:59.888820 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.888784 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:43:59.967379 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.967344 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.967648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.967612 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.967648 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.967642 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.967798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.967657 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:43:59.967798 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:43:59.967671 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25znp\" (UniqueName: \"kubernetes.io/projected/615e3dd4-8283-414d-a199-aa4b4df5bd58-kube-api-access-25znp\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:44:00.248398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.248362 2554 generic.go:356] "Generic (PLEG): container finished" podID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerID="194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1" exitCode=0 Jul 31 22:44:00.248618 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.248427 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" Jul 31 22:44:00.248618 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.248447 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" event={"ID":"615e3dd4-8283-414d-a199-aa4b4df5bd58","Type":"ContainerDied","Data":"194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1"} Jul 31 22:44:00.248618 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.248501 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd" event={"ID":"615e3dd4-8283-414d-a199-aa4b4df5bd58","Type":"ContainerDied","Data":"c6c33af4cb4c4625d4749934a04d8fb10e9c2efbb329f9f1a807cdae5b5b1459"} Jul 31 22:44:00.248618 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.248529 2554 scope.go:117] "RemoveContainer" containerID="194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1" Jul 31 22:44:00.258268 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.258242 2554 scope.go:117] "RemoveContainer" containerID="c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1" Jul 31 22:44:00.266527 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.266497 2554 scope.go:117] "RemoveContainer" containerID="194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1" Jul 31 22:44:00.267128 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:44:00.267096 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1\": container with ID starting with 194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1 not found: ID does not exist" containerID="194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1" Jul 31 22:44:00.267233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.267171 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1"} err="failed to get container status \"194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1\": rpc error: code = NotFound desc = could not find container \"194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1\": container with ID starting with 194164e8fe465e53ff01efe6c7488ec0c9d65362d4b25d6eafb9bc1f9b1085a1 not found: ID does not exist" Jul 31 22:44:00.267233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.267200 2554 scope.go:117] "RemoveContainer" containerID="c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1" Jul 31 22:44:00.267486 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:44:00.267468 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1\": container with ID starting with c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1 not found: ID does not exist" containerID="c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1" Jul 31 22:44:00.267554 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.267495 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1"} err="failed to get container status \"c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1\": rpc error: code = NotFound desc = could not find container \"c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1\": container with ID starting with c6da0d8feaa0332ec4c46811faaf3e7212358374658cbc3875f1456901e73ab1 not found: ID does not exist" Jul 31 22:44:00.269358 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.269340 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd"] Jul 31 22:44:00.274340 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:00.274322 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-84098f-openshift-default-847945c5dd-z2dfd"] Jul 31 22:44:01.740040 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:01.740000 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:44:01.956814 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:01.956780 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" path="/var/lib/kubelet/pods/615e3dd4-8283-414d-a199-aa4b4df5bd58/volumes" Jul 31 22:44:12.313081 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:12.313042 2554 generic.go:356] "Generic (PLEG): container finished" podID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerID="0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734" exitCode=0 Jul 31 22:44:12.313646 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:12.313090 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" event={"ID":"4b8d2f96-dcc7-4e03-b2fe-081c1592168c","Type":"ContainerDied","Data":"0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734"} Jul 31 22:44:12.313735 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:12.313642 2554 scope.go:117] "RemoveContainer" containerID="0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734" Jul 31 22:44:12.314731 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:12.314702 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:12.314800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:12.314771 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:12.314841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:12.314804 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:13.238165 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:13.238126 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:44:13.320396 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:13.320358 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" event={"ID":"4b8d2f96-dcc7-4e03-b2fe-081c1592168c","Type":"ContainerStarted","Data":"cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b"} Jul 31 22:44:13.845684 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:13.845645 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:44:13.846741 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:13.846715 2554 patch_prober.go:28] interesting pod/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 31 22:44:13.846874 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:13.846775 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 31 22:44:14.845821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:14.845785 2554 patch_prober.go:28] interesting pod/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 31 22:44:14.846232 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:14.845843 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 31 22:44:15.845656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:15.845618 2554 patch_prober.go:28] interesting pod/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 31 22:44:15.845838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:15.845684 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 31 22:44:16.845020 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:16.844986 2554 patch_prober.go:28] interesting pod/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 31 22:44:16.845405 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:16.845042 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 31 22:44:17.845495 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:17.845458 2554 patch_prober.go:28] interesting pod/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 31 22:44:17.845925 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:17.845514 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 31 22:44:18.850056 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:18.850025 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:44:18.850503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:18.850393 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:44:18.851265 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:18.851245 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:44:23.205062 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.205023 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:23.566323 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.566234 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:23.853202 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853113 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn"] Jul 31 22:44:23.853494 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853481 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" Jul 31 22:44:23.853494 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853494 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" Jul 31 22:44:23.853628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853510 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" Jul 31 22:44:23.853628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853516 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" Jul 31 22:44:23.853628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853588 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" Jul 31 22:44:23.853628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.853608 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="615e3dd4-8283-414d-a199-aa4b4df5bd58" containerName="istio-proxy" Jul 31 22:44:23.856934 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.856914 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.859233 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.859207 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3326ad-openshift-default-dockercfg-596f5\"" Jul 31 22:44:23.871504 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.871481 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn"] Jul 31 22:44:23.881799 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.881774 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:23.996771 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.996732 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd15086-114f-40d1-8283-2b270779f155-istio-podinfo\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.996975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.996780 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-istio-token\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.996975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.996903 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-data\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.996975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.996955 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-envoy\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.997116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.996977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd15086-114f-40d1-8283-2b270779f155-istiod-ca-cert\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.997116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.997030 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-certs\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.997116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.997072 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-credential-socket\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.997116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.997096 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-socket\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:23.997266 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:23.997133 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx975\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-kube-api-access-tx975\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098571 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098527 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd15086-114f-40d1-8283-2b270779f155-istio-podinfo\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098767 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098583 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-istio-token\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098767 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098635 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-data\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098767 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098663 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-envoy\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098767 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098686 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd15086-114f-40d1-8283-2b270779f155-istiod-ca-cert\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098820 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-certs\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-credential-socket\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098921 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-socket\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.098975 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.098961 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx975\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-kube-api-access-tx975\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.099197 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.099179 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-data\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.099277 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.099253 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-certs\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.099386 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.099364 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-credential-socket\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.099465 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.099444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-socket\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.099573 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.099553 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd15086-114f-40d1-8283-2b270779f155-istiod-ca-cert\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.101002 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.100985 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-envoy\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.101528 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.101513 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd15086-114f-40d1-8283-2b270779f155-istio-podinfo\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.109188 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.109108 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-istio-token\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.109542 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.109523 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx975\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-kube-api-access-tx975\") pod \"e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.169748 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.169717 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:24.334882 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.334858 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn"] Jul 31 22:44:24.337243 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:44:24.337215 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd15086_114f_40d1_8283_2b270779f155.slice/crio-67576456ccba1a775b0a70ee5ccb4b1412e356805934947c7312deecd58b84d0 WatchSource:0}: Error finding container 67576456ccba1a775b0a70ee5ccb4b1412e356805934947c7312deecd58b84d0: Status 404 returned error can't find the container with id 67576456ccba1a775b0a70ee5ccb4b1412e356805934947c7312deecd58b84d0 Jul 31 22:44:24.339559 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.339521 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:24.339852 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.339826 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:24.339904 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.339884 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:24.375540 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.375513 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" event={"ID":"1dd15086-114f-40d1-8283-2b270779f155","Type":"ContainerStarted","Data":"67576456ccba1a775b0a70ee5ccb4b1412e356805934947c7312deecd58b84d0"} Jul 31 22:44:24.415011 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:24.414983 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:25.384868 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:25.384824 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" event={"ID":"1dd15086-114f-40d1-8283-2b270779f155","Type":"ContainerStarted","Data":"cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf"} Jul 31 22:44:25.406965 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:25.406911 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podStartSLOduration=2.406894283 podStartE2EDuration="2.406894283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:44:25.404710312 +0000 UTC m=+2689.969558896" watchObservedRunningTime="2026-07-31 22:44:25.406894283 +0000 UTC m=+2689.971742865" Jul 31 22:44:26.170947 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:26.170908 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:26.176069 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:26.176041 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:26.390494 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:26.390466 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:26.391408 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:26.391390 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:27.352077 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:27.352041 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:27.451322 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:27.451271 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:28.501971 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:28.501892 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:28.601744 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:28.601708 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:29.067923 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:29.067894 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:29.164433 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:29.164401 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:44:31.041649 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:31.041614 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:44:36.149507 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:36.149479 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:44:36.153480 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:36.153460 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:44:41.392091 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.392050 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 31 22:44:41.392517 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.392112 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 31 22:44:41.466391 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.466361 2554 generic.go:356] "Generic (PLEG): container finished" podID="1dd15086-114f-40d1-8283-2b270779f155" containerID="cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf" exitCode=0 Jul 31 22:44:41.466618 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.466439 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" event={"ID":"1dd15086-114f-40d1-8283-2b270779f155","Type":"ContainerDied","Data":"cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf"} Jul 31 22:44:41.466907 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.466890 2554 scope.go:117] "RemoveContainer" containerID="cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf" Jul 31 22:44:41.467883 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.467857 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:41.467947 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.467921 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:41.467988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:41.467949 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 31 22:44:42.473224 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:42.473194 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" event={"ID":"1dd15086-114f-40d1-8283-2b270779f155","Type":"ContainerStarted","Data":"76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b"} Jul 31 22:44:43.170150 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:43.170117 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:43.170326 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:43.170293 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 31 22:44:43.170373 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:43.170337 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 31 22:44:44.170354 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:44.170318 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 31 22:44:44.170791 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:44.170389 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 31 22:44:45.170244 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:45.170205 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 31 22:44:45.170433 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:45.170260 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 31 22:44:46.170490 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:46.170454 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 31 22:44:46.170913 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:46.170507 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 31 22:44:47.170658 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:47.170619 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 31 22:44:47.171075 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:47.170680 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 31 22:44:48.174090 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:48.174061 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:48.174474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:48.174414 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:48.174999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:48.174978 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:44:52.395514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:44:52.395467 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:45:00.366779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:45:00.366744 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:45:02.756992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:45:02.756957 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:45:06.012929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:45:06.012892 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:45:33.075848 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:45:33.075810 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:45:36.233492 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:45:36.233458 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:46:03.147028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:46:03.146936 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:46:18.542291 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:46:18.542257 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:46:30.443698 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:46:30.443656 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:46:45.338022 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:46:45.337988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:46:56.938810 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:46:56.938774 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:47:11.645476 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:11.645441 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:47:22.244701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:22.244668 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:47:28.646833 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:28.646747 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:47:29.862477 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:29.862441 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:47:35.036760 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:35.036721 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:47:35.137897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:35.137863 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:47:38.338677 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:47:38.338639 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:48:04.239497 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:48:04.239464 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:48:16.434126 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:48:16.434084 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:48:30.935980 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:48:30.935941 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:48:41.742985 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:48:41.742945 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:48:57.247117 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:48:57.247080 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:49:01.566593 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:01.566514 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:49:02.351660 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:02.351624 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:49:06.865701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:06.865668 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:49:09.056028 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:09.055991 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:49:10.479746 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:10.479716 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:49:11.913104 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:11.913073 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:49:36.188041 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:36.188006 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:49:36.192103 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:36.192067 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:49:45.232924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:45.232876 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:49:48.037879 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:49:48.037848 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:50:25.157705 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:50:25.157672 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:50:32.075149 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:50:32.075066 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:50:35.668792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:50:35.668760 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:51:04.739482 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:04.739445 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:51:09.525779 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:09.525735 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:51:10.724752 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:10.724719 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:51:12.421901 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:12.421863 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:51:12.550150 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:12.550118 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:51:15.571882 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:15.571845 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:51:23.614145 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:23.614107 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:51:26.092425 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:26.092384 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:51:42.989359 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:42.989324 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:51:45.552297 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:51:45.552263 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:52:23.048801 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:52:23.048768 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:52:31.350530 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:52:31.350487 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:52:48.149047 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:52:48.149012 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:52:53.321821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:52:53.321777 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:52:54.330076 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:52:54.330040 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:52:56.539807 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:52:56.539767 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:53:01.131991 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.131953 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7485-140247624416400"] Jul 31 22:53:01.135507 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.135488 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:01.138157 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.138134 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5dbjw\"" Jul 31 22:53:01.143865 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.143845 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7485-140247624416400"] Jul 31 22:53:01.314786 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.314747 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gdjl\" (UniqueName: \"kubernetes.io/projected/5147c58e-fd60-40d8-9ffa-52d4cc0626e8-kube-api-access-6gdjl\") pod \"test-curl-7485-140247624416400\" (UID: \"5147c58e-fd60-40d8-9ffa-52d4cc0626e8\") " pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:01.415772 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.415682 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gdjl\" (UniqueName: \"kubernetes.io/projected/5147c58e-fd60-40d8-9ffa-52d4cc0626e8-kube-api-access-6gdjl\") pod \"test-curl-7485-140247624416400\" (UID: \"5147c58e-fd60-40d8-9ffa-52d4cc0626e8\") " pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:01.435275 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.435248 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gdjl\" (UniqueName: \"kubernetes.io/projected/5147c58e-fd60-40d8-9ffa-52d4cc0626e8-kube-api-access-6gdjl\") pod \"test-curl-7485-140247624416400\" (UID: \"5147c58e-fd60-40d8-9ffa-52d4cc0626e8\") " pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:01.445639 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.445615 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:01.579201 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.579172 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7485-140247624416400"] Jul 31 22:53:01.582345 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:53:01.582305 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5147c58e_fd60_40d8_9ffa_52d4cc0626e8.slice/crio-eefc14c646e0c524866c01e112c2b20f92be22fd618d799da26e3c41399c14ec WatchSource:0}: Error finding container eefc14c646e0c524866c01e112c2b20f92be22fd618d799da26e3c41399c14ec: Status 404 returned error can't find the container with id eefc14c646e0c524866c01e112c2b20f92be22fd618d799da26e3c41399c14ec Jul 31 22:53:01.583905 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:01.583883 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 31 22:53:02.018182 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:02.018141 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7485-140247624416400" event={"ID":"5147c58e-fd60-40d8-9ffa-52d4cc0626e8","Type":"ContainerStarted","Data":"eefc14c646e0c524866c01e112c2b20f92be22fd618d799da26e3c41399c14ec"} Jul 31 22:53:03.024751 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:03.024719 2554 generic.go:356] "Generic (PLEG): container finished" podID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" containerID="ec8979aae669e61f6abb8df838043eb12e2b77918f5d3838c0ecf0a4a227283f" exitCode=0 Jul 31 22:53:03.025130 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:03.024805 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7485-140247624416400" event={"ID":"5147c58e-fd60-40d8-9ffa-52d4cc0626e8","Type":"ContainerDied","Data":"ec8979aae669e61f6abb8df838043eb12e2b77918f5d3838c0ecf0a4a227283f"} Jul 31 22:53:04.172676 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.172648 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:04.193553 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.193526 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7485-140247624416400_5147c58e-fd60-40d8-9ffa-52d4cc0626e8/test-curl-7485-140247624416400/0.log" Jul 31 22:53:04.216847 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.216813 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7485-140247624416400"] Jul 31 22:53:04.222553 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.222523 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7485-140247624416400"] Jul 31 22:53:04.236101 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.236064 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gdjl\" (UniqueName: \"kubernetes.io/projected/5147c58e-fd60-40d8-9ffa-52d4cc0626e8-kube-api-access-6gdjl\") pod \"5147c58e-fd60-40d8-9ffa-52d4cc0626e8\" (UID: \"5147c58e-fd60-40d8-9ffa-52d4cc0626e8\") " Jul 31 22:53:04.238395 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.238370 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5147c58e-fd60-40d8-9ffa-52d4cc0626e8-kube-api-access-6gdjl" (OuterVolumeSpecName: "kube-api-access-6gdjl") pod "5147c58e-fd60-40d8-9ffa-52d4cc0626e8" (UID: "5147c58e-fd60-40d8-9ffa-52d4cc0626e8"). InnerVolumeSpecName "kube-api-access-6gdjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:53:04.337581 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:04.337494 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gdjl\" (UniqueName: \"kubernetes.io/projected/5147c58e-fd60-40d8-9ffa-52d4cc0626e8-kube-api-access-6gdjl\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:53:05.035935 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:05.035904 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:05.036113 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:05.035949 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eefc14c646e0c524866c01e112c2b20f92be22fd618d799da26e3c41399c14ec" Jul 31 22:53:05.046082 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:53:05.046055 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7485-140247624416400\" is forbidden: User \"system:node:ip-10-0-140-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-197.ec2.internal' and this object" podUID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:05.955246 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:53:05.955210 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7485-140247624416400\" is forbidden: User \"system:node:ip-10-0-140-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-197.ec2.internal' and this object" podUID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" pod="opendatahub/test-curl-7485-140247624416400" Jul 31 22:53:05.956929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:05.956901 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" path="/var/lib/kubelet/pods/5147c58e-fd60-40d8-9ffa-52d4cc0626e8/volumes" Jul 31 22:53:13.603701 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:13.603664 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:13.945457 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:13.945431 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:14.296935 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:14.296848 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:19.914361 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:19.914275 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:20.003112 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:20.003082 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:21.244181 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:21.244146 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:21.517372 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:21.517284 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:21.740356 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:21.740319 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:21.841166 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:21.841093 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:23.474741 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:23.474708 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:23.572477 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:23.572445 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:23.857373 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:23.857298 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:53:44.923038 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:44.922945 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:45.651298 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:45.651262 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:46.003760 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:46.003726 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:46.782277 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:46.782245 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:49.076661 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:49.076628 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:49.175950 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:49.175915 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:50.275993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:50.275960 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:50.579857 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:50.579764 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:51.103155 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.103119 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:51.383512 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.383428 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488"] Jul 31 22:53:51.384074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.384051 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" containerName="test-curl-7485-140247624416400" Jul 31 22:53:51.384120 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.384078 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" containerName="test-curl-7485-140247624416400" Jul 31 22:53:51.384223 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.384210 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5147c58e-fd60-40d8-9ffa-52d4cc0626e8" containerName="test-curl-7485-140247624416400" Jul 31 22:53:51.387660 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.387640 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.390837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.390813 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-918fe5-dockercfg-8nlxv\"" Jul 31 22:53:51.391363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.390876 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-918fe5\"" Jul 31 22:53:51.401361 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.401336 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:53:51.410287 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.410263 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488"] Jul 31 22:53:51.465671 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.465639 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50c5d81-2764-4e99-8bdc-3f459c36930d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-918fe5-5954df964-tt488\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.465841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.465773 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7dv9\" (UniqueName: \"kubernetes.io/projected/e50c5d81-2764-4e99-8bdc-3f459c36930d-kube-api-access-x7dv9\") pod \"payload-processing-e2e-ipp-b-918fe5-5954df964-tt488\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.567177 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.567137 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7dv9\" (UniqueName: \"kubernetes.io/projected/e50c5d81-2764-4e99-8bdc-3f459c36930d-kube-api-access-x7dv9\") pod \"payload-processing-e2e-ipp-b-918fe5-5954df964-tt488\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.567365 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.567206 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50c5d81-2764-4e99-8bdc-3f459c36930d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-918fe5-5954df964-tt488\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.567895 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.567877 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50c5d81-2764-4e99-8bdc-3f459c36930d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-918fe5-5954df964-tt488\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.575650 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.575592 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7dv9\" (UniqueName: \"kubernetes.io/projected/e50c5d81-2764-4e99-8bdc-3f459c36930d-kube-api-access-x7dv9\") pod \"payload-processing-e2e-ipp-b-918fe5-5954df964-tt488\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:51.700085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:51.700059 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:52.047706 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:52.047678 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488"] Jul 31 22:53:52.050097 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:53:52.050065 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50c5d81_2764_4e99_8bdc_3f459c36930d.slice/crio-d894e44a9cb8ce59d0a988076a4b516eba558990ce9115b7e456eaefac777ba7 WatchSource:0}: Error finding container d894e44a9cb8ce59d0a988076a4b516eba558990ce9115b7e456eaefac777ba7: Status 404 returned error can't find the container with id d894e44a9cb8ce59d0a988076a4b516eba558990ce9115b7e456eaefac777ba7 Jul 31 22:53:52.274474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:52.274436 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" event={"ID":"e50c5d81-2764-4e99-8bdc-3f459c36930d","Type":"ContainerStarted","Data":"d894e44a9cb8ce59d0a988076a4b516eba558990ce9115b7e456eaefac777ba7"} Jul 31 22:53:54.348068 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:54.348033 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:53:55.293133 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:55.293094 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" event={"ID":"e50c5d81-2764-4e99-8bdc-3f459c36930d","Type":"ContainerStarted","Data":"39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37"} Jul 31 22:53:55.293314 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:55.293161 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:53:55.310116 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:53:55.310070 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" podStartSLOduration=2.007158887 podStartE2EDuration="4.310054593s" totalImagesPullingTime="2.302895706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:53:55.308966535 +0000 UTC m=+3259.873815117" watchObservedRunningTime="2026-07-31 22:53:55.310054593 +0000 UTC m=+3259.874903182" Jul 31 22:54:06.301397 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:06.301366 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:54:14.773069 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:14.773032 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:54:21.261555 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:21.261525 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-918fe5-5954df964-tt488_e50c5d81-2764-4e99-8bdc-3f459c36930d/payload-processing/0.log" Jul 31 22:54:26.280786 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:26.280731 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:54:28.950531 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:28.950492 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:54:36.224083 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:36.224054 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:54:36.228452 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:36.228426 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:54:54.950958 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:54:54.950916 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:55:06.418726 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:06.418648 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-918fe5-5954df964-tt488_e50c5d81-2764-4e99-8bdc-3f459c36930d/payload-processing/0.log" Jul 31 22:55:06.705518 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:06.705489 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:07.046176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:07.046086 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:07.123781 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:07.123747 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:10.430538 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:10.430505 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:10.527556 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:10.527521 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:10.859375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:10.859330 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:11.586366 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:11.586326 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:11.776940 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:11.776900 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:12.204806 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.204772 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:12.307722 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.307688 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:12.354459 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.354422 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k"] Jul 31 22:55:12.358062 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.358037 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.360532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.360509 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-c5b030\"" Jul 31 22:55:12.373743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.373712 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k"] Jul 31 22:55:12.423999 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.423967 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl"] Jul 31 22:55:12.427426 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.427407 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.430018 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.429996 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-c5b030-dockercfg-l7lnn\"" Jul 31 22:55:12.438128 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.438104 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl"] Jul 31 22:55:12.512560 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.512482 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd269b11-ef56-4cc0-973a-fe10a18afe78-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.512560 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.512522 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn6mw\" (UniqueName: \"kubernetes.io/projected/bd269b11-ef56-4cc0-973a-fe10a18afe78-kube-api-access-pn6mw\") pod \"payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.614007 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.613966 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.614410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.614088 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd269b11-ef56-4cc0-973a-fe10a18afe78-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.614410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.614151 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn6mw\" (UniqueName: \"kubernetes.io/projected/bd269b11-ef56-4cc0-973a-fe10a18afe78-kube-api-access-pn6mw\") pod \"payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.614410 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.614196 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bc58\" (UniqueName: \"kubernetes.io/projected/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-kube-api-access-7bc58\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.614838 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.614820 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd269b11-ef56-4cc0-973a-fe10a18afe78-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.628658 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.628630 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn6mw\" (UniqueName: \"kubernetes.io/projected/bd269b11-ef56-4cc0-973a-fe10a18afe78-kube-api-access-pn6mw\") pod \"payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.668514 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.668478 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:12.721330 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.715654 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.721330 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.715858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bc58\" (UniqueName: \"kubernetes.io/projected/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-kube-api-access-7bc58\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.721330 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.717172 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.725148 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.725120 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bc58\" (UniqueName: \"kubernetes.io/projected/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-kube-api-access-7bc58\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.739285 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.739248 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:12.821400 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.821357 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k"] Jul 31 22:55:12.823586 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:55:12.823552 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd269b11_ef56_4cc0_973a_fe10a18afe78.slice/crio-c295e25aa367024a43325623712ba7deccff145b09e36a066a7655642f133471 WatchSource:0}: Error finding container c295e25aa367024a43325623712ba7deccff145b09e36a066a7655642f133471: Status 404 returned error can't find the container with id c295e25aa367024a43325623712ba7deccff145b09e36a066a7655642f133471 Jul 31 22:55:12.890767 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:12.890635 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl"] Jul 31 22:55:12.892845 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:55:12.892816 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074c9bc1_be48_47ab_9c0f_97e0a316b7a1.slice/crio-aeb07db391cd6eeaa89cb370c3e6408ca1e86ddaca937a17533724cbdf812fc9 WatchSource:0}: Error finding container aeb07db391cd6eeaa89cb370c3e6408ca1e86ddaca937a17533724cbdf812fc9: Status 404 returned error can't find the container with id aeb07db391cd6eeaa89cb370c3e6408ca1e86ddaca937a17533724cbdf812fc9 Jul 31 22:55:13.693689 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.693640 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" event={"ID":"074c9bc1-be48-47ab-9c0f-97e0a316b7a1","Type":"ContainerStarted","Data":"ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded"} Jul 31 22:55:13.694176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.693705 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" event={"ID":"074c9bc1-be48-47ab-9c0f-97e0a316b7a1","Type":"ContainerStarted","Data":"aeb07db391cd6eeaa89cb370c3e6408ca1e86ddaca937a17533724cbdf812fc9"} Jul 31 22:55:13.694176 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.693872 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:13.695423 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.695398 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" event={"ID":"bd269b11-ef56-4cc0-973a-fe10a18afe78","Type":"ContainerStarted","Data":"ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591"} Jul 31 22:55:13.695574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.695437 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" event={"ID":"bd269b11-ef56-4cc0-973a-fe10a18afe78","Type":"ContainerStarted","Data":"c295e25aa367024a43325623712ba7deccff145b09e36a066a7655642f133471"} Jul 31 22:55:13.695574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.695541 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:13.712349 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.712286 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" podStartSLOduration=1.7122725920000001 podStartE2EDuration="1.712272592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:55:13.709715873 +0000 UTC m=+3338.274564496" watchObservedRunningTime="2026-07-31 22:55:13.712272592 +0000 UTC m=+3338.277121174" Jul 31 22:55:13.727953 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:13.727912 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" podStartSLOduration=1.727902497 podStartE2EDuration="1.727902497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:55:13.725667008 +0000 UTC m=+3338.290515589" watchObservedRunningTime="2026-07-31 22:55:13.727902497 +0000 UTC m=+3338.292751079" Jul 31 22:55:17.546700 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:17.546663 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:55:24.703794 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:24.703763 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:24.704190 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:24.703933 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:35.485897 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:35.485862 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:42.389487 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.389453 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:42.430743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.430705 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k"] Jul 31 22:55:42.431007 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.430976 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" podUID="bd269b11-ef56-4cc0-973a-fe10a18afe78" containerName="payload-processing" containerID="cri-o://ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591" gracePeriod=30 Jul 31 22:55:42.439274 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.438884 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl"] Jul 31 22:55:42.439274 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.439133 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" podUID="074c9bc1-be48-47ab-9c0f-97e0a316b7a1" containerName="payload-pre-processing" containerID="cri-o://ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded" gracePeriod=30 Jul 31 22:55:42.472916 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.472885 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:42.549992 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.549965 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:42.620451 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.619775 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:42.728776 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.728751 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:42.731688 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.731666 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:42.803818 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.803784 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-plugins-config-volume\") pod \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " Jul 31 22:55:42.803971 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.803834 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn6mw\" (UniqueName: \"kubernetes.io/projected/bd269b11-ef56-4cc0-973a-fe10a18afe78-kube-api-access-pn6mw\") pod \"bd269b11-ef56-4cc0-973a-fe10a18afe78\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " Jul 31 22:55:42.803971 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.803886 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd269b11-ef56-4cc0-973a-fe10a18afe78-plugins-config-volume\") pod \"bd269b11-ef56-4cc0-973a-fe10a18afe78\" (UID: \"bd269b11-ef56-4cc0-973a-fe10a18afe78\") " Jul 31 22:55:42.803971 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.803911 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bc58\" (UniqueName: \"kubernetes.io/projected/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-kube-api-access-7bc58\") pod \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\" (UID: \"074c9bc1-be48-47ab-9c0f-97e0a316b7a1\") " Jul 31 22:55:42.804284 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.804254 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "074c9bc1-be48-47ab-9c0f-97e0a316b7a1" (UID: "074c9bc1-be48-47ab-9c0f-97e0a316b7a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:55:42.804363 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.804336 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd269b11-ef56-4cc0-973a-fe10a18afe78-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd269b11-ef56-4cc0-973a-fe10a18afe78" (UID: "bd269b11-ef56-4cc0-973a-fe10a18afe78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:55:42.806099 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.806075 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-kube-api-access-7bc58" (OuterVolumeSpecName: "kube-api-access-7bc58") pod "074c9bc1-be48-47ab-9c0f-97e0a316b7a1" (UID: "074c9bc1-be48-47ab-9c0f-97e0a316b7a1"). InnerVolumeSpecName "kube-api-access-7bc58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:55:42.806099 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.806090 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd269b11-ef56-4cc0-973a-fe10a18afe78-kube-api-access-pn6mw" (OuterVolumeSpecName: "kube-api-access-pn6mw") pod "bd269b11-ef56-4cc0-973a-fe10a18afe78" (UID: "bd269b11-ef56-4cc0-973a-fe10a18afe78"). InnerVolumeSpecName "kube-api-access-pn6mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:55:42.870687 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.870650 2554 generic.go:356] "Generic (PLEG): container finished" podID="074c9bc1-be48-47ab-9c0f-97e0a316b7a1" containerID="ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded" exitCode=0 Jul 31 22:55:42.870859 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.870698 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" Jul 31 22:55:42.870859 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.870737 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" event={"ID":"074c9bc1-be48-47ab-9c0f-97e0a316b7a1","Type":"ContainerDied","Data":"ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded"} Jul 31 22:55:42.870859 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.870799 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl" event={"ID":"074c9bc1-be48-47ab-9c0f-97e0a316b7a1","Type":"ContainerDied","Data":"aeb07db391cd6eeaa89cb370c3e6408ca1e86ddaca937a17533724cbdf812fc9"} Jul 31 22:55:42.870859 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.870829 2554 scope.go:117] "RemoveContainer" containerID="ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded" Jul 31 22:55:42.872155 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.872134 2554 generic.go:356] "Generic (PLEG): container finished" podID="bd269b11-ef56-4cc0-973a-fe10a18afe78" containerID="ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591" exitCode=0 Jul 31 22:55:42.872255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.872177 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" Jul 31 22:55:42.872255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.872193 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" event={"ID":"bd269b11-ef56-4cc0-973a-fe10a18afe78","Type":"ContainerDied","Data":"ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591"} Jul 31 22:55:42.872255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.872248 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k" event={"ID":"bd269b11-ef56-4cc0-973a-fe10a18afe78","Type":"ContainerDied","Data":"c295e25aa367024a43325623712ba7deccff145b09e36a066a7655642f133471"} Jul 31 22:55:42.882213 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.882196 2554 scope.go:117] "RemoveContainer" containerID="ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded" Jul 31 22:55:42.882560 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:55:42.882533 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded\": container with ID starting with ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded not found: ID does not exist" containerID="ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded" Jul 31 22:55:42.882678 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.882570 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded"} err="failed to get container status \"ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded\": rpc error: code = NotFound desc = could not find container \"ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded\": container with ID starting with ce2361b74698829c9a52934e07fe2294eb06b3aa5fc33be025ffa6ccf3dbcded not found: ID does not exist" Jul 31 22:55:42.882678 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.882593 2554 scope.go:117] "RemoveContainer" containerID="ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591" Jul 31 22:55:42.895227 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.895205 2554 scope.go:117] "RemoveContainer" containerID="ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591" Jul 31 22:55:42.895881 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:55:42.895849 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591\": container with ID starting with ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591 not found: ID does not exist" containerID="ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591" Jul 31 22:55:42.896001 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.895890 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591"} err="failed to get container status \"ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591\": rpc error: code = NotFound desc = could not find container \"ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591\": container with ID starting with ff609af500ea4798cfd44cebe250245d152d023ae77ecba21ac3892693940591 not found: ID does not exist" Jul 31 22:55:42.896069 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.896041 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl"] Jul 31 22:55:42.903372 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.903320 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5b030-56cf9dbcd4-ns4rl"] Jul 31 22:55:42.904719 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.904699 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-plugins-config-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:55:42.904768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.904726 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn6mw\" (UniqueName: \"kubernetes.io/projected/bd269b11-ef56-4cc0-973a-fe10a18afe78-kube-api-access-pn6mw\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:55:42.904810 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.904773 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd269b11-ef56-4cc0-973a-fe10a18afe78-plugins-config-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:55:42.904810 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.904789 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bc58\" (UniqueName: \"kubernetes.io/projected/074c9bc1-be48-47ab-9c0f-97e0a316b7a1-kube-api-access-7bc58\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:55:42.914972 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.914949 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k"] Jul 31 22:55:42.917106 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.917087 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-c5b030-596869759d-zdj5k"] Jul 31 22:55:42.997456 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:42.997413 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:43.957431 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:43.957393 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="074c9bc1-be48-47ab-9c0f-97e0a316b7a1" path="/var/lib/kubelet/pods/074c9bc1-be48-47ab-9c0f-97e0a316b7a1/volumes" Jul 31 22:55:43.957976 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:43.957955 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd269b11-ef56-4cc0-973a-fe10a18afe78" path="/var/lib/kubelet/pods/bd269b11-ef56-4cc0-973a-fe10a18afe78/volumes" Jul 31 22:55:45.847418 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:45.847383 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:55:49.116009 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:49.115975 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:49.182794 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:49.182759 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:49.399455 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:49.399374 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:49.637052 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:49.637022 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:55:53.223673 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.223637 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv"] Jul 31 22:55:53.224057 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.224023 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd269b11-ef56-4cc0-973a-fe10a18afe78" containerName="payload-processing" Jul 31 22:55:53.224057 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.224035 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd269b11-ef56-4cc0-973a-fe10a18afe78" containerName="payload-processing" Jul 31 22:55:53.224057 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.224051 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="074c9bc1-be48-47ab-9c0f-97e0a316b7a1" containerName="payload-pre-processing" Jul 31 22:55:53.224057 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.224056 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="074c9bc1-be48-47ab-9c0f-97e0a316b7a1" containerName="payload-pre-processing" Jul 31 22:55:53.224188 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.224125 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd269b11-ef56-4cc0-973a-fe10a18afe78" containerName="payload-processing" Jul 31 22:55:53.224188 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.224135 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="074c9bc1-be48-47ab-9c0f-97e0a316b7a1" containerName="payload-pre-processing" Jul 31 22:55:53.227273 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.227257 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:53.229884 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.229861 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lgfz6\"" Jul 31 22:55:53.230010 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.229903 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 31 22:55:53.236435 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.236411 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv"] Jul 31 22:55:53.401437 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.401403 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv\" (UID: \"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:53.502285 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.502206 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv\" (UID: \"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:53.502932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.502914 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv\" (UID: \"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:53.539132 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.539097 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:53.691231 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.691204 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv"] Jul 31 22:55:53.693376 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:55:53.693347 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d84c98_f7c3_4d92_a7d0_5edc7a1c67c7.slice/crio-71b4725eacbe706d78d21fb95b470b016e90e5d7d38c8a0924c1e2968e2c5222 WatchSource:0}: Error finding container 71b4725eacbe706d78d21fb95b470b016e90e5d7d38c8a0924c1e2968e2c5222: Status 404 returned error can't find the container with id 71b4725eacbe706d78d21fb95b470b016e90e5d7d38c8a0924c1e2968e2c5222 Jul 31 22:55:53.938849 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:53.938816 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" event={"ID":"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7","Type":"ContainerStarted","Data":"71b4725eacbe706d78d21fb95b470b016e90e5d7d38c8a0924c1e2968e2c5222"} Jul 31 22:55:54.945320 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:54.945285 2554 generic.go:356] "Generic (PLEG): container finished" podID="12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7" containerID="5dbc28ed928b3c19f3ddddc84d1f0920522c709b142398b9ad9ade9448dc3f1c" exitCode=0 Jul 31 22:55:54.945784 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:54.945384 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" event={"ID":"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7","Type":"ContainerDied","Data":"5dbc28ed928b3c19f3ddddc84d1f0920522c709b142398b9ad9ade9448dc3f1c"} Jul 31 22:55:56.092551 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.092528 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:56.124696 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.124664 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7-maas-api-service-ca\") pod \"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7\" (UID: \"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7\") " Jul 31 22:55:56.124982 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.124959 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7" (UID: "12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:55:56.226230 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.226152 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7-maas-api-service-ca\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:55:56.957477 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.957452 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" Jul 31 22:55:56.957682 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.957496 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv" event={"ID":"12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7","Type":"ContainerDied","Data":"71b4725eacbe706d78d21fb95b470b016e90e5d7d38c8a0924c1e2968e2c5222"} Jul 31 22:55:56.957682 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:55:56.957541 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71b4725eacbe706d78d21fb95b470b016e90e5d7d38c8a0924c1e2968e2c5222" Jul 31 22:56:06.738239 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:06.738197 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:56:07.028841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:07.028764 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:08.867026 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:08.866983 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:08.937303 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:08.937269 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:09.036218 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:09.036175 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:09.062924 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:09.062885 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv"] Jul 31 22:56:09.069258 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:09.069228 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-56c69a-cl2lv"] Jul 31 22:56:09.137398 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:09.137315 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:09.450580 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:09.450535 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:09.956789 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:09.956760 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7" path="/var/lib/kubelet/pods/12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7/volumes" Jul 31 22:56:14.527929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:14.527893 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:15.352039 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:15.352005 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:56:50.650120 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:50.650087 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:51.934847 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:51.934814 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:56:57.556932 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:56:57.556896 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:03.571134 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:03.571098 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:03.671497 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:03.671459 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:03.875312 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:03.875223 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:04.117286 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:04.117245 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:14.809986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:14.809945 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:14.848503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:14.848465 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488"] Jul 31 22:57:14.848799 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:14.848767 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" podUID="e50c5d81-2764-4e99-8bdc-3f459c36930d" containerName="payload-processing" containerID="cri-o://39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37" gracePeriod=30 Jul 31 22:57:14.910578 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:14.910509 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:15.009375 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.009342 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:15.073991 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.073906 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:15.115510 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.115484 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:57:15.264526 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.264489 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50c5d81-2764-4e99-8bdc-3f459c36930d-plugins-config-volume\") pod \"e50c5d81-2764-4e99-8bdc-3f459c36930d\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " Jul 31 22:57:15.264747 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.264545 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7dv9\" (UniqueName: \"kubernetes.io/projected/e50c5d81-2764-4e99-8bdc-3f459c36930d-kube-api-access-x7dv9\") pod \"e50c5d81-2764-4e99-8bdc-3f459c36930d\" (UID: \"e50c5d81-2764-4e99-8bdc-3f459c36930d\") " Jul 31 22:57:15.264929 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.264905 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50c5d81-2764-4e99-8bdc-3f459c36930d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e50c5d81-2764-4e99-8bdc-3f459c36930d" (UID: "e50c5d81-2764-4e99-8bdc-3f459c36930d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:57:15.266723 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.266704 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50c5d81-2764-4e99-8bdc-3f459c36930d-kube-api-access-x7dv9" (OuterVolumeSpecName: "kube-api-access-x7dv9") pod "e50c5d81-2764-4e99-8bdc-3f459c36930d" (UID: "e50c5d81-2764-4e99-8bdc-3f459c36930d"). InnerVolumeSpecName "kube-api-access-x7dv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:57:15.357550 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.357454 2554 generic.go:356] "Generic (PLEG): container finished" podID="e50c5d81-2764-4e99-8bdc-3f459c36930d" containerID="39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37" exitCode=0 Jul 31 22:57:15.357550 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.357518 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" Jul 31 22:57:15.357743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.357545 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" event={"ID":"e50c5d81-2764-4e99-8bdc-3f459c36930d","Type":"ContainerDied","Data":"39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37"} Jul 31 22:57:15.357743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.357619 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488" event={"ID":"e50c5d81-2764-4e99-8bdc-3f459c36930d","Type":"ContainerDied","Data":"d894e44a9cb8ce59d0a988076a4b516eba558990ce9115b7e456eaefac777ba7"} Jul 31 22:57:15.357743 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.357650 2554 scope.go:117] "RemoveContainer" containerID="39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37" Jul 31 22:57:15.365501 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.365477 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e50c5d81-2764-4e99-8bdc-3f459c36930d-plugins-config-volume\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:15.365615 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.365508 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7dv9\" (UniqueName: \"kubernetes.io/projected/e50c5d81-2764-4e99-8bdc-3f459c36930d-kube-api-access-x7dv9\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:15.366674 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.366649 2554 scope.go:117] "RemoveContainer" containerID="39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37" Jul 31 22:57:15.366935 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:57:15.366916 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37\": container with ID starting with 39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37 not found: ID does not exist" containerID="39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37" Jul 31 22:57:15.366987 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.366942 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37"} err="failed to get container status \"39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37\": rpc error: code = NotFound desc = could not find container \"39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37\": container with ID starting with 39effd46f42fec5a78ada7f3ccacd9e2b7e348716a241f6e2d1573da90970e37 not found: ID does not exist" Jul 31 22:57:15.380100 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.380072 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488"] Jul 31 22:57:15.392444 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.387229 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-918fe5-5954df964-tt488"] Jul 31 22:57:15.438143 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.438104 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:15.956989 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:15.956951 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50c5d81-2764-4e99-8bdc-3f459c36930d" path="/var/lib/kubelet/pods/e50c5d81-2764-4e99-8bdc-3f459c36930d/volumes" Jul 31 22:57:16.254995 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:16.254906 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:57:21.635260 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:21.635222 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:21.780172 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:21.780132 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:21.975731 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:21.975695 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:22.206813 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:22.206777 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:33.028474 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:33.028440 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:33.104498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:33.104466 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:33.207826 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:33.207788 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:33.308958 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:33.308874 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:33.623645 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:33.623526 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:35.152565 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:35.152533 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:57:39.711907 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:39.711874 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:39.812666 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:39.812626 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:40.039461 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:40.039378 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd"] Jul 31 22:57:40.039628 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:40.039592 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" containerID="cri-o://cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b" gracePeriod=30 Jul 31 22:57:40.069819 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:40.069785 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:40.252292 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:40.252256 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:45.297731 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.297704 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:57:45.440154 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440124 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-envoy\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440308 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440171 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-data\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440308 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440205 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-token\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440308 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440262 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9r6x\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-kube-api-access-j9r6x\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440308 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440288 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-certs\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440515 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440316 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-podinfo\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440515 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440359 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-credential-socket\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440515 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440417 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-socket\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440515 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440447 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istiod-ca-cert\") pod \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\" (UID: \"4b8d2f96-dcc7-4e03-b2fe-081c1592168c\") " Jul 31 22:57:45.440715 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440515 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:57:45.440828 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440796 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:57:45.440891 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440812 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.440891 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440847 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:57:45.441007 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.440914 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:57:45.441065 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.441024 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:57:45.442511 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.442483 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:57:45.442656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.442557 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:57:45.442656 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.442578 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-kube-api-access-j9r6x" (OuterVolumeSpecName: "kube-api-access-j9r6x") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "kube-api-access-j9r6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:57:45.442764 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.442680 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b8d2f96-dcc7-4e03-b2fe-081c1592168c" (UID: "4b8d2f96-dcc7-4e03-b2fe-081c1592168c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:57:45.512800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.512767 2554 generic.go:356] "Generic (PLEG): container finished" podID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerID="cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b" exitCode=0 Jul 31 22:57:45.512988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.512818 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" Jul 31 22:57:45.512988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.512854 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" event={"ID":"4b8d2f96-dcc7-4e03-b2fe-081c1592168c","Type":"ContainerDied","Data":"cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b"} Jul 31 22:57:45.512988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.512908 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd" event={"ID":"4b8d2f96-dcc7-4e03-b2fe-081c1592168c","Type":"ContainerDied","Data":"0dcd6e9b387493fdf459056a2c71260fad1109f53a16ffae39a1ee440b896488"} Jul 31 22:57:45.512988 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.512931 2554 scope.go:117] "RemoveContainer" containerID="cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b" Jul 31 22:57:45.522037 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.522018 2554 scope.go:117] "RemoveContainer" containerID="0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734" Jul 31 22:57:45.529792 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.529775 2554 scope.go:117] "RemoveContainer" containerID="cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b" Jul 31 22:57:45.530062 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:57:45.530045 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b\": container with ID starting with cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b not found: ID does not exist" containerID="cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b" Jul 31 22:57:45.530106 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.530071 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b"} err="failed to get container status \"cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b\": rpc error: code = NotFound desc = could not find container \"cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b\": container with ID starting with cfb8c4d726d0f122499791388eb681680120acf7fc3b553399725054bbf9d27b not found: ID does not exist" Jul 31 22:57:45.530106 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.530090 2554 scope.go:117] "RemoveContainer" containerID="0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734" Jul 31 22:57:45.530325 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:57:45.530307 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734\": container with ID starting with 0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734 not found: ID does not exist" containerID="0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734" Jul 31 22:57:45.530385 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.530334 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734"} err="failed to get container status \"0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734\": rpc error: code = NotFound desc = could not find container \"0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734\": container with ID starting with 0cd55bb092a74cf2e4548e0d583e63b8dd5a051878000d3a6dff19b1b75eb734 not found: ID does not exist" Jul 31 22:57:45.535074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.535050 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd"] Jul 31 22:57:45.539538 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.539519 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-08271f-openshift-default-5bfcccb844-vcnkd"] Jul 31 22:57:45.542006 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.541984 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9r6x\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-kube-api-access-j9r6x\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542010 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542025 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542037 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542050 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542063 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542085 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542075 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.542290 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.542088 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b8d2f96-dcc7-4e03-b2fe-081c1592168c-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:57:45.958292 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:45.958262 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" path="/var/lib/kubelet/pods/4b8d2f96-dcc7-4e03-b2fe-081c1592168c/volumes" Jul 31 22:57:51.040617 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:51.040562 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:51.144390 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:51.144358 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:51.216787 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:51.216759 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:51.288286 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:51.288251 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:51.713839 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:51.713803 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:53.057050 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:53.057002 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-xjr6m"] Jul 31 22:57:58.217537 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:58.217456 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:57:58.536074 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:58.535992 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn"] Jul 31 22:57:58.536269 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:58.536238 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" containerID="cri-o://76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b" gracePeriod=30 Jul 31 22:57:59.371048 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:57:59.371007 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:03.176561 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.176514 2554 patch_prober.go:28] interesting pod/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 31 22:58:03.177021 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.176627 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 31 22:58:03.786169 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.786144 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:58:03.832331 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832255 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd15086-114f-40d1-8283-2b270779f155-istio-podinfo\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832331 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832324 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-data\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832361 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-istio-token\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832388 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-certs\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832406 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx975\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-kube-api-access-tx975\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832440 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd15086-114f-40d1-8283-2b270779f155-istiod-ca-cert\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832466 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-credential-socket\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832532 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832499 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-socket\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832843 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832544 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-envoy\") pod \"1dd15086-114f-40d1-8283-2b270779f155\" (UID: \"1dd15086-114f-40d1-8283-2b270779f155\") " Jul 31 22:58:03.832843 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832695 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-data" (OuterVolumeSpecName: "istio-data") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:58:03.832843 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832732 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:58:03.832993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832869 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-data\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.832993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832913 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-certs\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.832993 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832929 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:58:03.833123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.832994 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dd15086-114f-40d1-8283-2b270779f155-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 31 22:58:03.833123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.833073 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:58:03.834754 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.834727 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-kube-api-access-tx975" (OuterVolumeSpecName: "kube-api-access-tx975") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "kube-api-access-tx975". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:58:03.834867 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.834749 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-istio-token" (OuterVolumeSpecName: "istio-token") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 31 22:58:03.834913 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.834878 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 31 22:58:03.835061 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.835041 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1dd15086-114f-40d1-8283-2b270779f155-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1dd15086-114f-40d1-8283-2b270779f155" (UID: "1dd15086-114f-40d1-8283-2b270779f155"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 31 22:58:03.934097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934063 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-workload-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.934097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934096 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-istio-envoy\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.934097 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934107 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd15086-114f-40d1-8283-2b270779f155-istio-podinfo\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.934339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934115 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-istio-token\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.934339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934125 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx975\" (UniqueName: \"kubernetes.io/projected/1dd15086-114f-40d1-8283-2b270779f155-kube-api-access-tx975\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.934339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934133 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd15086-114f-40d1-8283-2b270779f155-istiod-ca-cert\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:03.934339 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:03.934145 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd15086-114f-40d1-8283-2b270779f155-credential-socket\") on node \"ip-10-0-140-197.ec2.internal\" DevicePath \"\"" Jul 31 22:58:04.611866 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.611829 2554 generic.go:356] "Generic (PLEG): container finished" podID="1dd15086-114f-40d1-8283-2b270779f155" containerID="76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b" exitCode=0 Jul 31 22:58:04.612467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.611880 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" event={"ID":"1dd15086-114f-40d1-8283-2b270779f155","Type":"ContainerDied","Data":"76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b"} Jul 31 22:58:04.612467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.611885 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" Jul 31 22:58:04.612467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.611925 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn" event={"ID":"1dd15086-114f-40d1-8283-2b270779f155","Type":"ContainerDied","Data":"67576456ccba1a775b0a70ee5ccb4b1412e356805934947c7312deecd58b84d0"} Jul 31 22:58:04.612467 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.611946 2554 scope.go:117] "RemoveContainer" containerID="76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b" Jul 31 22:58:04.620853 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.620835 2554 scope.go:117] "RemoveContainer" containerID="cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf" Jul 31 22:58:04.628503 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.628483 2554 scope.go:117] "RemoveContainer" containerID="76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b" Jul 31 22:58:04.628791 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:58:04.628765 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b\": container with ID starting with 76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b not found: ID does not exist" containerID="76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b" Jul 31 22:58:04.628900 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.628804 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b"} err="failed to get container status \"76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b\": rpc error: code = NotFound desc = could not find container \"76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b\": container with ID starting with 76f7b36a182e2526fe9adc6970091f22b7bc857a6b039589e7e34529794a2f9b not found: ID does not exist" Jul 31 22:58:04.628900 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.628830 2554 scope.go:117] "RemoveContainer" containerID="cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf" Jul 31 22:58:04.629072 ip-10-0-140-197 kubenswrapper[2554]: E0731 22:58:04.629056 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf\": container with ID starting with cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf not found: ID does not exist" containerID="cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf" Jul 31 22:58:04.629123 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.629080 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf"} err="failed to get container status \"cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf\": rpc error: code = NotFound desc = could not find container \"cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf\": container with ID starting with cd004e8efb0f5f4610933197cee09a579f07d1e81819dc47613b1e20114b2cdf not found: ID does not exist" Jul 31 22:58:04.630589 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.630570 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn"] Jul 31 22:58:04.634721 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:04.634700 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3326ad-openshift-default-5987c69b44-9v8hn"] Jul 31 22:58:05.092837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:05.092808 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8s5pj_546e2ef6-5b6d-4baa-86e9-af4af32fb997/manager/0.log" Jul 31 22:58:05.214429 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:05.214395 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9h5zc_da540a84-c3e2-489d-a0d3-4a2815f955d2/manager/0.log" Jul 31 22:58:05.478800 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:05.478774 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mq52q_e8108e4d-cfa1-4504-b680-19f86559bc4d/server/0.log" Jul 31 22:58:05.722188 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:05.722153 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79bdc6db47-hgzgr_688d7818-2128-46e4-a0ca-8d4f3c4ec619/copy-manifests/0.log" Jul 31 22:58:05.729202 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:05.729135 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79bdc6db47-hgzgr_688d7818-2128-46e4-a0ca-8d4f3c4ec619/manager/0.log" Jul 31 22:58:05.956728 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:05.956689 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dd15086-114f-40d1-8283-2b270779f155" path="/var/lib/kubelet/pods/1dd15086-114f-40d1-8283-2b270779f155/volumes" Jul 31 22:58:06.501320 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:06.501288 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29758935-hv9gg_4fedcf1c-a50f-4b5d-a606-47892e30d78a/cleanup/0.log" Jul 31 22:58:07.822035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:07.821997 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-fmrd5_7c8c7e6b-3d3c-44f4-8029-72db538db29f/manager/0.log" Jul 31 22:58:08.200558 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:08.200529 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-rgbzl_35cc624c-c12c-4dab-b550-64f61496176d/manager/0.log" Jul 31 22:58:08.324255 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:08.324229 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-xjr6m_55ab41b6-21a2-4c19-93cf-2d3414f8f1b9/limitador/0.log" Jul 31 22:58:08.822442 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:08.822416 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2_09b0c9c6-1a5b-43d0-aa14-5d1bb6497817/istio-proxy/0.log" Jul 31 22:58:09.071453 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:09.071423 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c56dfbf59-9n74l_40e3a03e-cfad-4716-89c7-170297098365/kube-auth-proxy/0.log" Jul 31 22:58:09.320545 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:09.320515 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/1.log" Jul 31 22:58:09.689286 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:09.689259 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644f8b4f6-vsfsr_07c72711-0eaf-4064-b343-3572c0783bf0/router/0.log" Jul 31 22:58:10.417164 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:10.417134 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqdzlj_a40d0c80-1b73-4d0a-b8bd-488e3392eff0/main/0.log" Jul 31 22:58:16.430832 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:16.430797 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:16.532820 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:16.532780 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:19.268131 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:19.268076 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hkdcf_a473fec7-53b1-4440-b7cf-5cdaf810c897/global-pull-secret-syncer/0.log" Jul 31 22:58:19.321350 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:19.321321 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" Jul 31 22:58:19.490491 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:19.490456 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-197.ec2.internal_01e690b2a67d9f68121efde86b8c3543/haproxy/0.log" Jul 31 22:58:24.909948 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:24.909915 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-fmrd5_7c8c7e6b-3d3c-44f4-8029-72db538db29f/manager/0.log" Jul 31 22:58:25.058521 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:25.058486 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-rgbzl_35cc624c-c12c-4dab-b550-64f61496176d/manager/0.log" Jul 31 22:58:25.082035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:25.082006 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-xjr6m_55ab41b6-21a2-4c19-93cf-2d3414f8f1b9/limitador/0.log" Jul 31 22:58:26.267294 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:26.267265 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5rxkg_d67851f3-1f5a-44f0-838e-0279d191776c/konnectivity-agent/0.log" limit=1024 Jul 31 22:58:26.918335 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:26.918297 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-h4d99_22deacc6-c170-4286-9dab-f20d65fe111b/cluster-monitoring-operator/0.log" Jul 31 22:58:27.179763 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:27.179687 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2zbn5_3293c148-9e45-49dd-924e-3f899cb51143/node-exporter/0.log" Jul 31 22:58:27.206232 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:27.206210 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2zbn5_3293c148-9e45-49dd-924e-3f899cb51143/kube-rbac-proxy/0.log" Jul 31 22:58:27.235573 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:27.235546 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2zbn5_3293c148-9e45-49dd-924e-3f899cb51143/init-textfile/0.log" Jul 31 22:58:28.085025 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.084995 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-86rkq_fc1f2e38-6147-48be-809e-2ab089aefd8e/prometheus-operator-admission-webhook/0.log" Jul 31 22:58:28.747959 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.747926 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88"] Jul 31 22:58:28.748328 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748308 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50c5d81-2764-4e99-8bdc-3f459c36930d" containerName="payload-processing" Jul 31 22:58:28.748328 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748324 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50c5d81-2764-4e99-8bdc-3f459c36930d" containerName="payload-processing" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748341 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748349 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748365 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748374 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748402 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7" containerName="revoke-keys" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748410 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7" containerName="revoke-keys" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748473 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b8d2f96-dcc7-4e03-b2fe-081c1592168c" containerName="istio-proxy" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748480 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50c5d81-2764-4e99-8bdc-3f459c36930d" containerName="payload-processing" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748492 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" Jul 31 22:58:28.748498 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748500 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dd15086-114f-40d1-8283-2b270779f155" containerName="istio-proxy" Jul 31 22:58:28.748847 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.748506 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="12d84c98-f7c3-4d92-a7d0-5edc7a1c67c7" containerName="revoke-keys" Jul 31 22:58:28.751837 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.751821 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.754860 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.754841 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rch42\"/\"kube-root-ca.crt\"" Jul 31 22:58:28.754962 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.754844 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rch42\"/\"openshift-service-ca.crt\"" Jul 31 22:58:28.755803 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.755780 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rch42\"/\"default-dockercfg-4qg57\"" Jul 31 22:58:28.762641 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.762621 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88"] Jul 31 22:58:28.862762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.862721 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvbxj\" (UniqueName: \"kubernetes.io/projected/94e0a028-31a9-43f2-b960-6040bd7ca841-kube-api-access-fvbxj\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.862762 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.862753 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-sys\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.863022 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.862777 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-podres\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.863022 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.862804 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-proc\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.863022 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.862900 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-lib-modules\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.963574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963539 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-podres\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.963768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963583 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-proc\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.963768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963646 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-lib-modules\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.963768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963699 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvbxj\" (UniqueName: \"kubernetes.io/projected/94e0a028-31a9-43f2-b960-6040bd7ca841-kube-api-access-fvbxj\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.963768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963716 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-sys\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.963768 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963756 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-podres\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.964014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-sys\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.964014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963762 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-proc\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.964014 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.963817 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94e0a028-31a9-43f2-b960-6040bd7ca841-lib-modules\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:28.988915 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:28.988885 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvbxj\" (UniqueName: \"kubernetes.io/projected/94e0a028-31a9-43f2-b960-6040bd7ca841-kube-api-access-fvbxj\") pod \"perf-node-gather-daemonset-8qb88\" (UID: \"94e0a028-31a9-43f2-b960-6040bd7ca841\") " pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:29.063121 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:29.063056 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:29.202037 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:29.202012 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88"] Jul 31 22:58:29.204014 ip-10-0-140-197 kubenswrapper[2554]: W0731 22:58:29.203986 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod94e0a028_31a9_43f2_b960_6040bd7ca841.slice/crio-f14be547d0e7b63009f48d0ecc68d83dc236a33ea20cfe034f318e8f93dfd864 WatchSource:0}: Error finding container f14be547d0e7b63009f48d0ecc68d83dc236a33ea20cfe034f318e8f93dfd864: Status 404 returned error can't find the container with id f14be547d0e7b63009f48d0ecc68d83dc236a33ea20cfe034f318e8f93dfd864 Jul 31 22:58:29.707712 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:29.707688 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-4bdbz_cf66033c-d967-4619-a9d3-bf7b399653b1/networking-console-plugin/0.log" Jul 31 22:58:29.746841 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:29.746807 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" event={"ID":"94e0a028-31a9-43f2-b960-6040bd7ca841","Type":"ContainerStarted","Data":"71becadd136595e7b11f47417715b60bc463c73fad73d5dfafc06cec6ee36235"} Jul 31 22:58:29.747035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:29.746858 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:29.747035 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:29.746876 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" event={"ID":"94e0a028-31a9-43f2-b960-6040bd7ca841","Type":"ContainerStarted","Data":"f14be547d0e7b63009f48d0ecc68d83dc236a33ea20cfe034f318e8f93dfd864"} Jul 31 22:58:30.332371 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:30.332341 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/1.log" Jul 31 22:58:30.335970 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:30.335949 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-bf69l_730d0f93-98f3-42a8-9ad2-a153a2cd71f9/console-operator/2.log" Jul 31 22:58:30.892639 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:30.892586 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-46znc_91e66e36-c614-401b-b57d-3cfd4257849e/download-server/0.log" Jul 31 22:58:31.512543 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:31.512513 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-dk522_1c31e4e4-71ec-4ce6-9b4f-cc004da9f007/volume-data-source-validator/0.log" Jul 31 22:58:32.606448 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:32.606417 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4466l_f4f6cbf1-c75d-4c44-b103-0cac8cb1480e/dns/0.log" Jul 31 22:58:32.635318 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:32.635282 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4466l_f4f6cbf1-c75d-4c44-b103-0cac8cb1480e/kube-rbac-proxy/0.log" Jul 31 22:58:32.861350 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:32.861272 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xhptq_2ed8dd22-e69c-44e5-8184-1694658e7c85/dns-node-resolver/0.log" Jul 31 22:58:33.550821 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:33.550792 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vt8r7_b107a90e-c7a2-4c47-8fdc-d0a48ef8bde7/node-ca/0.log" Jul 31 22:58:34.633404 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:34.633368 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2_09b0c9c6-1a5b-43d0-aa14-5d1bb6497817/istio-proxy/0.log" Jul 31 22:58:34.852715 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:34.852688 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c56dfbf59-9n74l_40e3a03e-cfad-4716-89c7-170297098365/kube-auth-proxy/0.log" Jul 31 22:58:34.916563 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:34.916477 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/0.log" Jul 31 22:58:34.922585 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:34.922565 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/1.log" Jul 31 22:58:35.015394 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:35.015366 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644f8b4f6-vsfsr_07c72711-0eaf-4064-b343-3572c0783bf0/router/0.log" Jul 31 22:58:35.761690 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:35.761663 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" Jul 31 22:58:35.782096 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:35.782049 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rch42/perf-node-gather-daemonset-8qb88" podStartSLOduration=7.782033384 podStartE2EDuration="7.782033384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:29.768788279 +0000 UTC m=+3534.333636861" watchObservedRunningTime="2026-07-31 22:58:35.782033384 +0000 UTC m=+3540.346881965" Jul 31 22:58:35.848756 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:35.848728 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6cj9w_22807a73-4c34-4adc-9993-5b3019dbd876/serve-healthcheck-canary/0.log" Jul 31 22:58:36.375951 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:36.375923 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-m67tr_5b3d2cae-4201-4a92-abab-c15ff7865fbf/insights-operator/0.log" Jul 31 22:58:36.376911 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:36.376893 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-m67tr_5b3d2cae-4201-4a92-abab-c15ff7865fbf/insights-operator/1.log" Jul 31 22:58:36.403072 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:36.403046 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7tmn9_d3f3f02b-f0e4-4ef6-8ab7-3663abde1396/kube-rbac-proxy/0.log" Jul 31 22:58:36.428867 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:36.428842 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7tmn9_d3f3f02b-f0e4-4ef6-8ab7-3663abde1396/exporter/0.log" Jul 31 22:58:36.456586 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:36.456555 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7tmn9_d3f3f02b-f0e4-4ef6-8ab7-3663abde1396/extractor/0.log" Jul 31 22:58:39.078574 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.078545 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8s5pj_546e2ef6-5b6d-4baa-86e9-af4af32fb997/manager/0.log" Jul 31 22:58:39.300286 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.300253 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9h5zc_da540a84-c3e2-489d-a0d3-4a2815f955d2/manager/0.log" Jul 31 22:58:39.380122 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.380038 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2_09b0c9c6-1a5b-43d0-aa14-5d1bb6497817/istio-proxy/0.log" Jul 31 22:58:39.509776 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.509750 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mq52q_e8108e4d-cfa1-4504-b680-19f86559bc4d/server/0.log" Jul 31 22:58:39.618487 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.618451 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c56dfbf59-9n74l_40e3a03e-cfad-4716-89c7-170297098365/kube-auth-proxy/0.log" Jul 31 22:58:39.634139 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.634030 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79bdc6db47-hgzgr_688d7818-2128-46e4-a0ca-8d4f3c4ec619/manager/0.log" Jul 31 22:58:39.658104 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.658073 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/0.log" Jul 31 22:58:39.661268 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.661242 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79bdc6db47-hgzgr_688d7818-2128-46e4-a0ca-8d4f3c4ec619/copy-manifests/0.log" Jul 31 22:58:39.664870 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.664851 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/1.log" Jul 31 22:58:39.779205 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:39.779174 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644f8b4f6-vsfsr_07c72711-0eaf-4064-b343-3572c0783bf0/router/0.log" Jul 31 22:58:41.208986 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:41.208938 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gmwx2_09b0c9c6-1a5b-43d0-aa14-5d1bb6497817/istio-proxy/0.log" Jul 31 22:58:41.389755 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:41.389728 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c56dfbf59-9n74l_40e3a03e-cfad-4716-89c7-170297098365/kube-auth-proxy/0.log" Jul 31 22:58:41.433209 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:41.433183 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/0.log" Jul 31 22:58:41.439361 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:41.439340 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hhkcc_b8d36c88-b636-43e4-a316-a9101dc08989/istio-proxy/1.log" Jul 31 22:58:41.501745 ip-10-0-140-197 kubenswrapper[2554]: I0731 22:58:41.501674 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-644f8b4f6-vsfsr_07c72711-0eaf-4064-b343-3572c0783bf0/router/0.log"