Jul 10 22:42:19.809942 ip-10-0-138-188 systemd[1]: Starting Kubernetes Kubelet... Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.271182 2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275595 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275616 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275620 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275623 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275627 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275631 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275634 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 22:42:20.450015 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275637 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275640 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275643 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275645 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275648 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275651 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275654 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275657 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275659 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275662 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275665 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275668 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275671 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275674 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275683 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275686 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275689 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275692 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275695 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275698 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 22:42:20.451564 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275701 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275704 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275706 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275708 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275711 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275714 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275716 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275719 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275721 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275724 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275727 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275730 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275733 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275736 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275738 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275741 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275744 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275746 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275749 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 22:42:20.455540 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275751 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275754 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275757 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275760 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275764 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275766 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275769 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275772 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275775 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275778 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275780 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275783 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275785 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275788 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275791 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275794 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275797 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275799 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275802 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275804 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 22:42:20.459237 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275807 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275812 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275816 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275819 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275822 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275825 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275829 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275833 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275837 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275840 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275843 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275846 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275848 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275851 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275853 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275856 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275859 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275862 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275864 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 22:42:20.464343 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275867 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275869 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275873 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275875 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275878 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275881 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275884 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275886 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275889 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275892 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275894 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275897 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275901 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.275904 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276707 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276716 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276720 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276723 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276726 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 22:42:20.466101 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276729 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276732 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276735 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276738 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276741 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276744 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276747 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276750 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276752 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276755 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276758 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276761 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276763 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276766 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276769 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276771 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276777 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276779 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276782 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276786 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 22:42:20.467167 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276789 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276792 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276795 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276798 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276802 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276807 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276810 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276814 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276817 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276820 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276823 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276826 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276829 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276832 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276835 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276838 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276840 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276844 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276847 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276849 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 22:42:20.468319 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276852 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276855 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276857 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276860 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276863 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276866 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276868 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276874 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276878 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276881 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276885 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276888 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276894 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276898 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276900 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276903 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276906 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276908 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276911 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 22:42:20.469306 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276914 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276917 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276919 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276922 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276927 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276929 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276932 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276935 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276938 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276940 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276944 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276947 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276949 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276952 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276955 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276957 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276960 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276962 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276965 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276968 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 22:42:20.470156 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276970 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276973 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276975 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276978 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276981 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276985 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276989 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276993 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276996 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.276999 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277001 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277004 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277006 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277009 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277011 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277403 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277409 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277413 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277415 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 22:42:20.470794 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277418 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277421 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277424 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277427 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277430 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277433 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277435 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277438 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277441 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277444 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277446 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277449 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277452 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277454 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277457 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277459 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277462 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277465 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277470 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277472 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 22:42:20.471442 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277475 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277477 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277480 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277482 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277485 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277487 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277490 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277493 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277495 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277499 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277502 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277504 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277507 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277510 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277513 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277515 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277518 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277520 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277523 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 22:42:20.472076 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277525 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277528 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277530 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277533 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277536 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277538 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277571 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277574 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277578 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277581 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277591 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277596 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277598 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277601 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277604 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277608 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277612 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277615 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277618 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277620 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 22:42:20.472710 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277623 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277625 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277630 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277633 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277635 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277638 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277641 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277643 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277646 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277649 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277651 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277654 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277656 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277660 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277664 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277667 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277670 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277673 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277676 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277679 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 22:42:20.473346 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277682 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277685 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277688 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277694 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277697 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277699 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277702 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277705 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277707 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277710 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277713 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277716 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277720 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277723 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277727 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.277729 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277804 2551 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277812 2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277819 2551 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277824 2551 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 22:42:20.474199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277828 2551 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277832 2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277836 2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277841 2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277844 2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277847 2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277851 2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277855 2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277858 2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277861 2551 flags.go:64] FLAG: --cgroup-root="" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277864 2551 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277867 2551 flags.go:64] FLAG: --client-ca-file="" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277871 2551 flags.go:64] FLAG: --cloud-provider="external" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277884 2551 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277889 2551 flags.go:64] FLAG: --cluster-domain="" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277893 2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277897 2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277901 2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277905 2551 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277909 2551 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277912 2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277915 2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277918 2551 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277921 2551 flags.go:64] FLAG: --contention-profiling="false" Jul 10 22:42:20.474886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277924 2551 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277928 2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277932 2551 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277936 2551 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277945 2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277949 2551 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277952 2551 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277955 2551 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277959 2551 flags.go:64] FLAG: --enable-server="true" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277962 2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277967 2551 flags.go:64] FLAG: --event-burst="100" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277970 2551 flags.go:64] FLAG: --event-qps="50" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277974 2551 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277977 2551 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277980 2551 flags.go:64] FLAG: --eviction-hard="" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277984 2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277987 2551 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277990 2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277993 2551 flags.go:64] FLAG: --eviction-soft="" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.277996 2551 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278000 2551 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278003 2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278006 2551 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278011 2551 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278014 2551 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 22:42:20.475931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278018 2551 flags.go:64] FLAG: --feature-gates="" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278022 2551 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278025 2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278028 2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278031 2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278035 2551 flags.go:64] FLAG: --healthz-port="10248" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278038 2551 flags.go:64] FLAG: --help="false" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278041 2551 flags.go:64] FLAG: --hostname-override="ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278044 2551 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278049 2551 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278052 2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278055 2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278059 2551 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278062 2551 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278065 2551 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278068 2551 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278071 2551 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278074 2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278077 2551 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278080 2551 flags.go:64] FLAG: --kube-reserved="" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278083 2551 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278086 2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278089 2551 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278093 2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 22:42:20.476781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278095 2551 flags.go:64] FLAG: --lock-file="" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278098 2551 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278101 2551 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278105 2551 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278110 2551 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278113 2551 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278118 2551 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278121 2551 flags.go:64] FLAG: --logging-format="text" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278124 2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278129 2551 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278132 2551 flags.go:64] FLAG: --manifest-url="" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278135 2551 flags.go:64] FLAG: --manifest-url-header="" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278139 2551 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278142 2551 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278146 2551 flags.go:64] FLAG: --max-pods="110" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278149 2551 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278152 2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278157 2551 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278161 2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278164 2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278167 2551 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278170 2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278177 2551 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278181 2551 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278184 2551 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 22:42:20.477857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278187 2551 flags.go:64] FLAG: --pod-cidr="" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278190 2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278197 2551 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278200 2551 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278204 2551 flags.go:64] FLAG: --pods-per-core="0" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278207 2551 flags.go:64] FLAG: --port="10250" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278210 2551 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278213 2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f88d6afb909fb73e" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278216 2551 flags.go:64] FLAG: --qos-reserved="" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278220 2551 flags.go:64] FLAG: --read-only-port="10255" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278223 2551 flags.go:64] FLAG: --register-node="true" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278226 2551 flags.go:64] FLAG: --register-with-taints="" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278230 2551 flags.go:64] FLAG: --registry-burst="10" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278235 2551 flags.go:64] FLAG: --registry-qps="5" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278238 2551 flags.go:64] FLAG: --reserved-cpus="" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278242 2551 flags.go:64] FLAG: --reserved-memory="" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278246 2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278250 2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278253 2551 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278256 2551 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278259 2551 flags.go:64] FLAG: --runonce="false" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278262 2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278265 2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278268 2551 flags.go:64] FLAG: --seccomp-default="false" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278271 2551 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 22:42:20.481375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278276 2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278279 2551 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278282 2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278285 2551 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278289 2551 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278292 2551 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278295 2551 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278298 2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278301 2551 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278304 2551 flags.go:64] FLAG: --system-cgroups="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278307 2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278313 2551 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278316 2551 flags.go:64] FLAG: --tls-cert-file="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278319 2551 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278323 2551 flags.go:64] FLAG: --tls-min-version="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278326 2551 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278329 2551 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278332 2551 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278335 2551 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278338 2551 flags.go:64] FLAG: --v="2" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278344 2551 flags.go:64] FLAG: --version="false" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278348 2551 flags.go:64] FLAG: --vmodule="" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278352 2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.278355 2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 22:42:20.482887 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278485 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278491 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278495 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278498 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278500 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278503 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278506 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278509 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278513 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278516 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278519 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278522 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278525 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278527 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278530 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278532 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278535 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278538 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278558 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278561 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 22:42:20.491330 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278564 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 22:42:20.484315 ip-10-0-138-188 systemd[1]: Started Kubernetes Kubelet. Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278567 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278570 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278573 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278576 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278579 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278582 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278584 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278588 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278591 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278594 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278597 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278600 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278603 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278607 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278610 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278613 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278616 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278618 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278621 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 22:42:20.492677 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278625 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278628 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278631 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278633 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278636 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278639 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278641 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278644 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278647 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278650 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278654 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278657 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278659 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278662 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278665 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278668 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278670 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278673 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278675 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278678 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 22:42:20.493514 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278682 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278685 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278687 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278690 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278693 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278695 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278698 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278701 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278703 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278707 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278711 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278714 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278718 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278721 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278724 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278727 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278730 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278733 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278735 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 22:42:20.494375 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278738 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278741 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278743 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278746 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278749 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278752 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278754 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278757 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278760 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278762 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278765 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278768 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278770 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278775 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278777 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278780 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278783 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278786 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278788 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 22:42:20.495464 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.278791 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.279733 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.291028 2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.291052 2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.291074 2551 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.291080 2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291144 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291150 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291155 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291158 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291161 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291166 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291170 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291173 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 22:42:20.496520 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291176 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291179 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291182 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291185 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291188 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291191 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291198 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291201 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291209 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291212 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291215 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291218 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291220 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291223 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291226 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291229 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291232 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291234 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291237 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291240 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 22:42:20.497407 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291243 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291246 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291249 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291251 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291255 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291257 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291260 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291263 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291266 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291268 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291271 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291273 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291277 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291280 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291282 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291285 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291288 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291291 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291296 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 22:42:20.498452 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291298 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291302 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291306 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291310 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291312 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291315 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291318 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291321 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291323 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291326 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291329 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291332 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291334 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291337 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291340 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291342 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291345 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291348 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291350 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291353 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 22:42:20.499602 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291356 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291358 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291362 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291365 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291368 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291371 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291374 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291377 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291380 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291383 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291385 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291390 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291393 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291396 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291399 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291402 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291405 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291407 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291410 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291413 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 22:42:20.500671 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291415 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291418 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291421 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291423 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291426 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291428 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291431 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291434 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291436 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291440 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291442 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291445 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.291451 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291592 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 22:42:20.501862 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291609 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291613 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291616 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291620 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291623 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291626 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291629 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291634 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291637 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291640 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291644 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291648 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291652 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291656 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291658 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291661 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291664 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291668 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291671 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291674 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 22:42:20.502482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291677 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291681 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291684 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291687 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291690 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291693 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291696 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291699 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291702 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291705 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291708 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291710 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291713 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291716 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291719 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291721 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291724 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291727 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291729 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 22:42:20.503352 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291733 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291736 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291738 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291743 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291746 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291749 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291752 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291755 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291757 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291760 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291763 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291766 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291768 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291771 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291774 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291776 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291779 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291781 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291784 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 22:42:20.504007 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291786 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291789 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291792 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291795 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291797 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291800 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291802 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291805 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291808 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291810 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291813 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291815 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291818 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291826 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291829 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291831 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291835 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291838 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291841 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291845 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 22:42:20.504856 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291847 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291850 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291852 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291855 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291858 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291861 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291863 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291866 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291869 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291871 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291875 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291877 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291880 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291882 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291885 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291888 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291890 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291893 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291897 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 22:42:20.505486 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:20.291900 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.291905 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.292058 2551 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.294531 2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.296147 2551 server.go:1019] "Starting client certificate rotation" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.296249 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.296286 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.299163 2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.300768 2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.314834 2551 log.go:25] "Validated CRI v1 runtime API" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.324600 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 22:42:20.609271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.370701 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-264gv" Jul 10 22:42:20.631967 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.375649 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-264gv" Jul 10 22:42:20.631967 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.442701 2551 log.go:25] "Validated CRI v1 image API" Jul 10 22:42:20.631967 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.443696 2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 22:42:20.631967 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.449509 2551 fs.go:135] Filesystem UUIDs: map[715a879b-f661-4cdd-ab19-bc164327b7c7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a7e24c1e-7b1e-4930-a18c-c1b8881f8a08:/dev/nvme0n1p3] Jul 10 22:42:20.631967 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.449557 2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.459868 2551 manager.go:216] Machine: {Timestamp:2026-07-10 22:42:20.458299234 +0000 UTC m=+0.513391069 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101759 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb7def538daecd2537dda9ff41e45 SystemUUID:ec2eb7de-f538-daec-d253-7dda9ff41e45 BootID:4fa24422-e1cb-40a6-91c5-9393c524af17 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b0:c3:38:c5:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b0:c3:38:c5:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:0a:76:98:cc:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.460025 2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.460144 2551 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.463702 2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.463760 2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.464585 2551 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.464603 2551 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.464611 2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.464632 2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.465917 2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470292 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470462 2551 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470497 2551 kubelet.go:486] "Attempting to sync node with API server" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470511 2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470526 2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470539 2551 kubelet.go:398] "Adding apiserver pod source" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.470569 2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 22:42:20.632808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.471064 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.471085 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.475758 2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.477764 2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.477804 2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478670 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478687 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478698 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478707 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478719 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478726 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478737 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478745 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478755 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478763 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478775 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478784 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478811 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.478819 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.483408 2551 server.go:1268] "Started kubelet" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.484008 2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.486716 2551 server.go:310] "Adding debug handlers to kubelet server" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.487449 2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.487510 2551 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.489105 2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.494321 2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.495871 2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.495984 2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.496248 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.497644 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.498087 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-188.ec2.internal\" not found" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.498849 2551 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.498875 2551 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.499017 2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.499111 2551 reconstruct.go:97] "Volume reconstruction finished" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.499121 2551 reconciler.go:29] "Reconciler: start to sync state" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.501244 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.501558 2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-188.ec2.internal" not found Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.501939 2551 factory.go:55] Registering systemd factory Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.501951 2551 factory.go:223] Registration of the systemd container factory successfully Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.502168 2551 factory.go:153] Registering CRI-O factory Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.502178 2551 factory.go:223] Registration of the crio container factory successfully Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.502225 2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.502244 2551 factory.go:103] Registering Raw factory Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.502257 2551 manager.go:1196] Started watching for new ooms in manager Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.502734 2551 manager.go:349] Starting recovery of all containers Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.508706 2551 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-188.ec2.internal\" not found" node="ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.517383 2551 manager.go:354] Recovery completed Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.521393 2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-188.ec2.internal" not found Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.521534 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.523640 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.635526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.523674 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.523689 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.524240 2551 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.524250 2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.524269 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.524313 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.525086 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.525112 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.525122 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.526363 2551 policy_none.go:49] "None policy: Start" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.526378 2551 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.526389 2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.528411 2551 policy_none.go:47] "Start" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.577806 2551 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-188.ec2.internal" not found Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.584295 2551 manager.go:337] "Starting Device Plugin manager" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.584344 2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.584357 2551 server.go:87] "Starting device plugin registration server" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.584696 2551 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.584708 2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.585103 2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.585174 2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.585181 2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.586132 2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.586187 2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-188.ec2.internal\" not found" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.616091 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.617444 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.617513 2551 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.617561 2551 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.617601 2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 22:42:20.637199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.620035 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.752302 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.752351 2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal"] Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.752420 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.755803 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.755834 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.755845 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.755867 2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.756270 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.756298 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.756308 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.758555 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.758663 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.758691 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.759322 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.759333 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.759345 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.759357 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.759356 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.759440 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.761652 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.761680 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.762421 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientMemory" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.762444 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.762455 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeHasSufficientPID" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.767165 2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:20.767190 2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-188.ec2.internal\": node \"ip-10-0-138-188.ec2.internal\" not found" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.783326 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:20.789641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.787009 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.309115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:20.799590 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.309115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.271044 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/76f2493457929e09a73d5d44d486b48d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" (UID: \"76f2493457929e09a73d5d44d486b48d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.309115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.271093 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76f2493457929e09a73d5d44d486b48d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" (UID: \"76f2493457929e09a73d5d44d486b48d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.309115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.271113 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a475e5bcf6b6d12915427e27f6213c9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-188.ec2.internal\" (UID: \"2a475e5bcf6b6d12915427e27f6213c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.313094 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.312680 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 22:42:22.313094 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.313002 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 22:42:22.316496 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.315680 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 22:42:22.316496 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:22.315729 2551 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-188.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.316496 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.315745 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.329134 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:22.329182 2551 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.371269 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/76f2493457929e09a73d5d44d486b48d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" (UID: \"76f2493457929e09a73d5d44d486b48d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.371310 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76f2493457929e09a73d5d44d486b48d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" (UID: \"76f2493457929e09a73d5d44d486b48d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.371329 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a475e5bcf6b6d12915427e27f6213c9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-188.ec2.internal\" (UID: \"2a475e5bcf6b6d12915427e27f6213c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.371378 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76f2493457929e09a73d5d44d486b48d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" (UID: \"76f2493457929e09a73d5d44d486b48d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.371387 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/76f2493457929e09a73d5d44d486b48d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal\" (UID: \"76f2493457929e09a73d5d44d486b48d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.484796 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.371424 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a475e5bcf6b6d12915427e27f6213c9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-188.ec2.internal\" (UID: \"2a475e5bcf6b6d12915427e27f6213c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.666832 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.615368 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.666832 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.617160 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" Jul 10 22:42:22.848960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.713327 2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 22:42:22.848960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.713488 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 22:42:22.848960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.713506 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 22:42:22.848960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.713509 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 22:42:22.848960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.798850 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 22:37:21 +0000 UTC" deadline="2028-02-05 04:51:50.790688281 +0000 UTC" Jul 10 22:42:22.848960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.798923 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13782h9m27.991767791s" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.891228 2551 apiserver.go:52] "Watching apiserver" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.899652 2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.900067 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-rx6mq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal","openshift-multus/multus-fggm7","openshift-network-diagnostics/network-check-target-jxrxm","openshift-network-operator/iptables-alerter-cgxzr","kube-system/konnectivity-agent-l5rk7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9","openshift-cluster-node-tuning-operator/tuned-sc978","openshift-multus/multus-additional-cni-plugins-p5d8f","openshift-multus/network-metrics-daemon-28nzx","openshift-ovn-kubernetes/ovnkube-node-5lnkm","kube-system/global-pull-secret-syncer-qgkfr","kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal","openshift-dns/node-resolver-wm5gq"] Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.902591 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.904654 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fggm7" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.904995 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.905056 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.905106 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.905103 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nq64j\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.906663 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.906810 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:22.906896 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.907100 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vg8hw\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.907136 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.907346 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.907431 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.908886 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.911440 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.911630 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.911721 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.911848 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9r2jp\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.912106 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.914299 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.914349 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zwrwq\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.914505 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.914655 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.916141 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.916895 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.916926 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.916971 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sx8xf\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.917085 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 22:42:22.917807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.917241 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 22:42:22.919749 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.919076 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:22.919749 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.919193 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:22.919749 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.919654 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jj2hk\"" Jul 10 22:42:22.919749 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.919665 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.922078 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.921274 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 22:42:22.922078 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.921308 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 22:42:22.922078 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.921404 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hgfnw\"" Jul 10 22:42:22.922078 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.921520 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:22.922078 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:22.921620 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:22.924667 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.923880 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:22.926853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.926140 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:22.926853 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:22.926225 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:22.926853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.926328 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 22:42:22.926853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.926573 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2czwh\"" Jul 10 22:42:22.926853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.926671 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 22:42:22.928792 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.926931 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.928792 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.927179 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 22:42:22.928792 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.927246 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 22:42:22.928792 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.927253 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 22:42:22.928792 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.928472 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:22.934812 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.931720 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m562j\"" Jul 10 22:42:22.934812 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.931911 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 22:42:22.934812 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.931968 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.937818 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.959721 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b5vg4" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.969482 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b5vg4" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974226 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d9861af-65ee-4e7b-94d6-1c44887a2d68-cni-binary-copy\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974261 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-k8s-cni-cncf-io\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974301 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-cni-bin\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974321 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-device-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974349 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e19610a-10bf-4e53-8a48-eb5bb7363d1a-agent-certs\") pod \"konnectivity-agent-l5rk7\" (UID: \"2e19610a-10bf-4e53-8a48-eb5bb7363d1a\") " pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974368 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysconfig\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974419 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-sys\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974435 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/516d62a3-1c72-4f8c-8e36-85b0ad205065-host-slash\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974458 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974474 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e19610a-10bf-4e53-8a48-eb5bb7363d1a-konnectivity-ca\") pod \"konnectivity-agent-l5rk7\" (UID: \"2e19610a-10bf-4e53-8a48-eb5bb7363d1a\") " pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974488 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-lib-modules\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974502 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f40b374-3519-464b-b29e-c3c2303f7fa8-serviceca\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974525 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-netns\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974561 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqt97\" (UniqueName: \"kubernetes.io/projected/516d62a3-1c72-4f8c-8e36-85b0ad205065-kube-api-access-bqt97\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.031979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974586 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-modprobe-d\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974602 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tqpw\" (UniqueName: \"kubernetes.io/projected/52f8726e-85dd-468b-9fa3-df73264a74d9-kube-api-access-4tqpw\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974628 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-kubelet\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974644 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-etc-kubernetes\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974660 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v28vl\" (UniqueName: \"kubernetes.io/projected/0d9861af-65ee-4e7b-94d6-1c44887a2d68-kube-api-access-v28vl\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974693 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-socket-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974719 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-registration-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974739 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974758 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f40b374-3519-464b-b29e-c3c2303f7fa8-host\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974785 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62hfm\" (UniqueName: \"kubernetes.io/projected/0f40b374-3519-464b-b29e-c3c2303f7fa8-kube-api-access-62hfm\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974801 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-cni-multus\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974825 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv9km\" (UniqueName: \"kubernetes.io/projected/a566dd78-47fc-444d-beee-60f784cf722d-kube-api-access-tv9km\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974840 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysctl-conf\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974863 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-var-lib-kubelet\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974880 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-tuned\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974894 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-system-cni-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.032794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974908 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-os-release\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974922 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-socket-dir-parent\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974936 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-etc-selinux\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974970 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysctl-d\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.974995 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-run\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975017 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52f8726e-85dd-468b-9fa3-df73264a74d9-tmp\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975034 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-cni-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975050 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-conf-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975064 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-daemon-config\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975079 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-multus-certs\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975093 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-sys-fs\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975113 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-systemd\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975132 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-cnibin\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975149 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-hostroot\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975178 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/516d62a3-1c72-4f8c-8e36-85b0ad205065-iptables-alerter-script\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975199 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-kubernetes\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:22.975235 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-host\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.034313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.018469 2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075476 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2jds\" (UniqueName: \"kubernetes.io/projected/fcf08e02-1e2e-470b-be75-c145cdd82140-kube-api-access-z2jds\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075513 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d9861af-65ee-4e7b-94d6-1c44887a2d68-cni-binary-copy\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075543 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-k8s-cni-cncf-io\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075606 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-k8s-cni-cncf-io\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075620 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-device-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075674 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e19610a-10bf-4e53-8a48-eb5bb7363d1a-agent-certs\") pod \"konnectivity-agent-l5rk7\" (UID: \"2e19610a-10bf-4e53-8a48-eb5bb7363d1a\") " pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075712 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysconfig\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075748 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-lib-modules\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075785 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtgkn\" (UniqueName: \"kubernetes.io/projected/2f6886d0-03e4-4074-9a3d-6105b77c91cc-kube-api-access-qtgkn\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075821 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n77d5\" (UniqueName: \"kubernetes.io/projected/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-kube-api-access-n77d5\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075855 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075889 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-os-release\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075908 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60803b6d-b5c5-4c66-b602-f4e7c229ce13-dbus\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075936 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-cni-netd\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075963 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-ovnkube-config\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.075987 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-netns\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.104865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076016 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-modprobe-d\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076049 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cnibin\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076079 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-hosts-file\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076109 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mwbm\" (UniqueName: \"kubernetes.io/projected/931fa3c6-353f-43b5-9016-bfae335b8cc7-kube-api-access-5mwbm\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076143 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-log-socket\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076176 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-socket-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076202 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-slash\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076252 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-device-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076261 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-cni-bin\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076312 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f40b374-3519-464b-b29e-c3c2303f7fa8-host\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076344 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62hfm\" (UniqueName: \"kubernetes.io/projected/0f40b374-3519-464b-b29e-c3c2303f7fa8-kube-api-access-62hfm\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076372 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-cni-multus\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076402 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysctl-conf\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076433 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-var-lib-kubelet\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076465 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-tuned\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076494 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-system-cni-dir\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076536 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-ovn\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.105931 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076572 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysconfig\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076572 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-system-cni-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076615 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-os-release\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076651 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076643 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-etc-selinux\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076694 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysctl-d\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076725 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-run\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076749 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-cni-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076771 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-conf-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076796 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-daemon-config\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076817 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/516d62a3-1c72-4f8c-8e36-85b0ad205065-iptables-alerter-script\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076829 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-modprobe-d\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076835 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-host\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076887 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-host\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076929 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcf08e02-1e2e-470b-be75-c145cdd82140-ovn-node-metrics-cert\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076951 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-cni-bin\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076970 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-sys\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076994 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-ovnkube-script-lib\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.112064 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077014 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/516d62a3-1c72-4f8c-8e36-85b0ad205065-host-slash\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077031 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e19610a-10bf-4e53-8a48-eb5bb7363d1a-konnectivity-ca\") pod \"konnectivity-agent-l5rk7\" (UID: \"2e19610a-10bf-4e53-8a48-eb5bb7363d1a\") " pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077048 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077076 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f40b374-3519-464b-b29e-c3c2303f7fa8-serviceca\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077110 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqt97\" (UniqueName: \"kubernetes.io/projected/516d62a3-1c72-4f8c-8e36-85b0ad205065-kube-api-access-bqt97\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077135 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tqpw\" (UniqueName: \"kubernetes.io/projected/52f8726e-85dd-468b-9fa3-df73264a74d9-kube-api-access-4tqpw\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077166 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-run-netns\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077203 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-run\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.076728 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-netns\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077263 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f40b374-3519-464b-b29e-c3c2303f7fa8-host\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077633 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-lib-modules\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077665 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-conf-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077673 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-cni-multus\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077720 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-cni-bin\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077730 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysctl-d\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077765 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-sys\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077202 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-run-ovn-kubernetes\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077796 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-sysctl-conf\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077814 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-kubelet\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077827 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-var-lib-kubelet\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077843 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-etc-kubernetes\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077850 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-system-cni-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077871 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v28vl\" (UniqueName: \"kubernetes.io/projected/0d9861af-65ee-4e7b-94d6-1c44887a2d68-kube-api-access-v28vl\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077897 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-registration-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077928 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077957 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.077986 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078011 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-tmp-dir\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078086 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-socket-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078150 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv9km\" (UniqueName: \"kubernetes.io/projected/a566dd78-47fc-444d-beee-60f784cf722d-kube-api-access-tv9km\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078183 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078222 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-kubelet\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078247 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-var-lib-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078275 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-etc-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.113816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078301 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078328 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-node-log\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078338 2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078375 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078400 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-socket-dir-parent\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078425 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52f8726e-85dd-468b-9fa3-df73264a74d9-tmp\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078447 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-env-overrides\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078471 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-multus-certs\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078506 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d9861af-65ee-4e7b-94d6-1c44887a2d68-cni-binary-copy\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078523 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-sys-fs\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078622 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-sys-fs\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078625 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-os-release\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078666 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2e19610a-10bf-4e53-8a48-eb5bb7363d1a-konnectivity-ca\") pod \"konnectivity-agent-l5rk7\" (UID: \"2e19610a-10bf-4e53-8a48-eb5bb7363d1a\") " pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078691 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-systemd\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078658 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-systemd\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078735 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-systemd\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.114865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078764 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078765 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-cni-dir\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078780 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-etc-selinux\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078798 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-cnibin\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078822 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-etc-kubernetes\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078826 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-hostroot\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078854 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-kubernetes\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078858 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-socket-dir-parent\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078692 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/516d62a3-1c72-4f8c-8e36-85b0ad205065-iptables-alerter-script\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078893 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60803b6d-b5c5-4c66-b602-f4e7c229ce13-kubelet-config\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078897 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-run-multus-certs\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078931 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-systemd-units\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078939 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-host-var-lib-kubelet\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078899 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a566dd78-47fc-444d-beee-60f784cf722d-registration-dir\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.078993 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/516d62a3-1c72-4f8c-8e36-85b0ad205065-host-slash\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.079057 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-cnibin\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.079099 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d9861af-65ee-4e7b-94d6-1c44887a2d68-hostroot\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.079237 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-kubernetes\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.115561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.079365 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f40b374-3519-464b-b29e-c3c2303f7fa8-serviceca\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.079385 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d9861af-65ee-4e7b-94d6-1c44887a2d68-multus-daemon-config\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.082112 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52f8726e-85dd-468b-9fa3-df73264a74d9-tmp\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.082161 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/52f8726e-85dd-468b-9fa3-df73264a74d9-etc-tuned\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.082462 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2e19610a-10bf-4e53-8a48-eb5bb7363d1a-agent-certs\") pod \"konnectivity-agent-l5rk7\" (UID: \"2e19610a-10bf-4e53-8a48-eb5bb7363d1a\") " pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.084892 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.084916 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.084932 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.085011 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:23.584974129 +0000 UTC m=+2.222399795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.085279 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62hfm\" (UniqueName: \"kubernetes.io/projected/0f40b374-3519-464b-b29e-c3c2303f7fa8-kube-api-access-62hfm\") pod \"node-ca-rx6mq\" (UID: \"0f40b374-3519-464b-b29e-c3c2303f7fa8\") " pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.086066 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqt97\" (UniqueName: \"kubernetes.io/projected/516d62a3-1c72-4f8c-8e36-85b0ad205065-kube-api-access-bqt97\") pod \"iptables-alerter-cgxzr\" (UID: \"516d62a3-1c72-4f8c-8e36-85b0ad205065\") " pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.086852 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tqpw\" (UniqueName: \"kubernetes.io/projected/52f8726e-85dd-468b-9fa3-df73264a74d9-kube-api-access-4tqpw\") pod \"tuned-sc978\" (UID: \"52f8726e-85dd-468b-9fa3-df73264a74d9\") " pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.087296 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v28vl\" (UniqueName: \"kubernetes.io/projected/0d9861af-65ee-4e7b-94d6-1c44887a2d68-kube-api-access-v28vl\") pod \"multus-fggm7\" (UID: \"0d9861af-65ee-4e7b-94d6-1c44887a2d68\") " pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.116635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.087371 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv9km\" (UniqueName: \"kubernetes.io/projected/a566dd78-47fc-444d-beee-60f784cf722d-kube-api-access-tv9km\") pod \"aws-ebs-csi-driver-node-4j6q9\" (UID: \"a566dd78-47fc-444d-beee-60f784cf722d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.146913 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.146889 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" Jul 10 22:42:23.180112 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180075 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcf08e02-1e2e-470b-be75-c145cdd82140-ovn-node-metrics-cert\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180112 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180115 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-ovnkube-script-lib\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180147 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180170 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-run-netns\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180192 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-run-ovn-kubernetes\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180243 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.180265 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180283 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-run-netns\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180268 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180293 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-run-ovn-kubernetes\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.180349 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:23.680327177 +0000 UTC m=+2.317752832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:23.180372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180373 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-tmp-dir\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180409 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180435 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-kubelet\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180447 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180460 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-var-lib-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180483 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-etc-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180504 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180538 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-node-log\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180570 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180593 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180621 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-env-overrides\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.180625 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180648 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-systemd\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180681 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180740 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-node-log\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180746 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.180816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180817 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-tmp-dir\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.180862 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:23.680843415 +0000 UTC m=+2.318269068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180870 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-var-lib-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180912 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-systemd\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180915 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-kubelet\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180682 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180958 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60803b6d-b5c5-4c66-b602-f4e7c229ce13-kubelet-config\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180985 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-systemd-units\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181013 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2jds\" (UniqueName: \"kubernetes.io/projected/fcf08e02-1e2e-470b-be75-c145cdd82140-kube-api-access-z2jds\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181022 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-systemd-units\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181026 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181030 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60803b6d-b5c5-4c66-b602-f4e7c229ce13-kubelet-config\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180959 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-etc-openvswitch\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.180984 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-ovnkube-script-lib\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181139 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-env-overrides\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181179 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtgkn\" (UniqueName: \"kubernetes.io/projected/2f6886d0-03e4-4074-9a3d-6105b77c91cc-kube-api-access-qtgkn\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.181553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181200 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n77d5\" (UniqueName: \"kubernetes.io/projected/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-kube-api-access-n77d5\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181266 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-os-release\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181322 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60803b6d-b5c5-4c66-b602-f4e7c229ce13-dbus\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181344 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-cni-netd\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181405 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-ovnkube-config\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181418 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-os-release\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181433 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cnibin\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181471 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-cni-netd\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181473 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-hosts-file\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181499 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cnibin\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181512 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwbm\" (UniqueName: \"kubernetes.io/projected/931fa3c6-353f-43b5-9016-bfae335b8cc7-kube-api-access-5mwbm\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181565 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-log-socket\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181568 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60803b6d-b5c5-4c66-b602-f4e7c229ce13-dbus\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181510 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181514 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-hosts-file\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181599 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-slash\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181626 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-log-socket\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.182340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181627 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-cni-bin\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181672 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-system-cni-dir\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181696 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-slash\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181699 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-ovn\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181711 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-host-cni-bin\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181729 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcf08e02-1e2e-470b-be75-c145cdd82140-run-ovn\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181775 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f6886d0-03e4-4074-9a3d-6105b77c91cc-system-cni-dir\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181863 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f6886d0-03e4-4074-9a3d-6105b77c91cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.183024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.181969 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcf08e02-1e2e-470b-be75-c145cdd82140-ovnkube-config\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.183268 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.183090 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcf08e02-1e2e-470b-be75-c145cdd82140-ovn-node-metrics-cert\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.190246 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.190196 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n77d5\" (UniqueName: \"kubernetes.io/projected/098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7-kube-api-access-n77d5\") pod \"node-resolver-wm5gq\" (UID: \"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7\") " pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.190814 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.190780 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mwbm\" (UniqueName: \"kubernetes.io/projected/931fa3c6-353f-43b5-9016-bfae335b8cc7-kube-api-access-5mwbm\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:23.190814 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.190789 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2jds\" (UniqueName: \"kubernetes.io/projected/fcf08e02-1e2e-470b-be75-c145cdd82140-kube-api-access-z2jds\") pod \"ovnkube-node-5lnkm\" (UID: \"fcf08e02-1e2e-470b-be75-c145cdd82140\") " pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.191236 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.191202 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sc978" Jul 10 22:42:23.191707 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.191685 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtgkn\" (UniqueName: \"kubernetes.io/projected/2f6886d0-03e4-4074-9a3d-6105b77c91cc-kube-api-access-qtgkn\") pod \"multus-additional-cni-plugins-p5d8f\" (UID: \"2f6886d0-03e4-4074-9a3d-6105b77c91cc\") " pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.213985 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.213962 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" Jul 10 22:42:23.226181 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.226156 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:23.230994 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.230974 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wm5gq" Jul 10 22:42:23.255074 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.255025 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rx6mq" Jul 10 22:42:23.295261 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.295191 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fggm7" Jul 10 22:42:23.309396 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.309363 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cgxzr" Jul 10 22:42:23.378832 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.378754 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:23.428555 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.428508 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f2493457929e09a73d5d44d486b48d.slice/crio-f4d2573647bab874888daf1f8fa3ba4d6850d7d0160ed9f3962ffc0726c99b96 WatchSource:0}: Error finding container f4d2573647bab874888daf1f8fa3ba4d6850d7d0160ed9f3962ffc0726c99b96: Status 404 returned error can't find the container with id f4d2573647bab874888daf1f8fa3ba4d6850d7d0160ed9f3962ffc0726c99b96 Jul 10 22:42:23.428840 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.428819 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a475e5bcf6b6d12915427e27f6213c9.slice/crio-433d385bcdfd5d6d5518d3c584121b919bb291a7396d2826715886d6aff6ded8 WatchSource:0}: Error finding container 433d385bcdfd5d6d5518d3c584121b919bb291a7396d2826715886d6aff6ded8: Status 404 returned error can't find the container with id 433d385bcdfd5d6d5518d3c584121b919bb291a7396d2826715886d6aff6ded8 Jul 10 22:42:23.434074 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.434047 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 22:42:23.567987 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.567953 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf08e02_1e2e_470b_be75_c145cdd82140.slice/crio-2fb68680fafd60eb02fd58a6dc749830e3f5a40b320f90e32c6e042719c21ef1 WatchSource:0}: Error finding container 2fb68680fafd60eb02fd58a6dc749830e3f5a40b320f90e32c6e042719c21ef1: Status 404 returned error can't find the container with id 2fb68680fafd60eb02fd58a6dc749830e3f5a40b320f90e32c6e042719c21ef1 Jul 10 22:42:23.585976 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.585938 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda566dd78_47fc_444d_beee_60f784cf722d.slice/crio-67e24a0b8eabbdb32b72399caf74bed57bc205f285c02fe6e1ee910c12b7726c WatchSource:0}: Error finding container 67e24a0b8eabbdb32b72399caf74bed57bc205f285c02fe6e1ee910c12b7726c: Status 404 returned error can't find the container with id 67e24a0b8eabbdb32b72399caf74bed57bc205f285c02fe6e1ee910c12b7726c Jul 10 22:42:23.619447 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.619415 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6886d0_03e4_4074_9a3d_6105b77c91cc.slice/crio-3e97864b04e76b66c1f438f6b303bdcb58315b69b36dad21a9188cd8886ab115 WatchSource:0}: Error finding container 3e97864b04e76b66c1f438f6b303bdcb58315b69b36dad21a9188cd8886ab115: Status 404 returned error can't find the container with id 3e97864b04e76b66c1f438f6b303bdcb58315b69b36dad21a9188cd8886ab115 Jul 10 22:42:23.620322 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.620298 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f40b374_3519_464b_b29e_c3c2303f7fa8.slice/crio-cead4d0ec5ec049f0cf47b2ade7a627a203665497a9393f8197294db9b3e67f9 WatchSource:0}: Error finding container cead4d0ec5ec049f0cf47b2ade7a627a203665497a9393f8197294db9b3e67f9: Status 404 returned error can't find the container with id cead4d0ec5ec049f0cf47b2ade7a627a203665497a9393f8197294db9b3e67f9 Jul 10 22:42:23.671312 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.671279 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516d62a3_1c72_4f8c_8e36_85b0ad205065.slice/crio-45a1ba4a7cc7a1b26665d3a9abbeee9b9578fdab932054fcf3476997647c81df WatchSource:0}: Error finding container 45a1ba4a7cc7a1b26665d3a9abbeee9b9578fdab932054fcf3476997647c81df: Status 404 returned error can't find the container with id 45a1ba4a7cc7a1b26665d3a9abbeee9b9578fdab932054fcf3476997647c81df Jul 10 22:42:23.685691 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.685660 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:23.685894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.685703 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:23.685894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.685734 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:23.685894 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685810 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:23.685894 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685860 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:23.685894 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685878 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:24.685863093 +0000 UTC m=+3.323288745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:23.685894 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685882 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:23.686126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685905 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:23.686126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685923 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:23.686126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685928 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:24.685890941 +0000 UTC m=+3.323316593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:23.686126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:23.685969 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:24.685954126 +0000 UTC m=+3.323379779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:23.692638 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.692609 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:23.696738 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.696713 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098d0cd2_46a5_4c65_85cc_8a3ee65bf3a7.slice/crio-967bca1ca985211c52b47eed1f799d0f11c09ceaed2c677da9d3d311b9a62eaa WatchSource:0}: Error finding container 967bca1ca985211c52b47eed1f799d0f11c09ceaed2c677da9d3d311b9a62eaa: Status 404 returned error can't find the container with id 967bca1ca985211c52b47eed1f799d0f11c09ceaed2c677da9d3d311b9a62eaa Jul 10 22:42:23.733374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.733336 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:23.788480 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.788455 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9861af_65ee_4e7b_94d6_1c44887a2d68.slice/crio-9c89d481ba249079cda9f4bd7738a0cff52864cd022831006b7458434d6c53bb WatchSource:0}: Error finding container 9c89d481ba249079cda9f4bd7738a0cff52864cd022831006b7458434d6c53bb: Status 404 returned error can't find the container with id 9c89d481ba249079cda9f4bd7738a0cff52864cd022831006b7458434d6c53bb Jul 10 22:42:23.848187 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.848159 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 22:42:23.862928 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.862900 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f8726e_85dd_468b_9fa3_df73264a74d9.slice/crio-602b74ea690b2ae01a7ab3f06ffb1914477ecd427d6daca8fe8e063c36bf5c16 WatchSource:0}: Error finding container 602b74ea690b2ae01a7ab3f06ffb1914477ecd427d6daca8fe8e063c36bf5c16: Status 404 returned error can't find the container with id 602b74ea690b2ae01a7ab3f06ffb1914477ecd427d6daca8fe8e063c36bf5c16 Jul 10 22:42:23.872647 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:23.872616 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e19610a_10bf_4e53_8a48_eb5bb7363d1a.slice/crio-9fc9d26bdfbbd18cb29391d885bc9f963a9c9f5e880ba57d9183eb171df3cf06 WatchSource:0}: Error finding container 9fc9d26bdfbbd18cb29391d885bc9f963a9c9f5e880ba57d9183eb171df3cf06: Status 404 returned error can't find the container with id 9fc9d26bdfbbd18cb29391d885bc9f963a9c9f5e880ba57d9183eb171df3cf06 Jul 10 22:42:23.973018 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.972880 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 22:37:22 +0000 UTC" deadline="2028-01-12 18:05:44.91161434 +0000 UTC" Jul 10 22:42:23.973018 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:23.972947 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13219h23m20.938669635s" Jul 10 22:42:24.035757 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.035719 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:24.035932 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.035773 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:24.035932 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.035906 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:24.036056 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.036026 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:24.040652 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.040602 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5rk7" event={"ID":"2e19610a-10bf-4e53-8a48-eb5bb7363d1a","Type":"ContainerStarted","Data":"9fc9d26bdfbbd18cb29391d885bc9f963a9c9f5e880ba57d9183eb171df3cf06"} Jul 10 22:42:24.042035 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.041981 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cgxzr" event={"ID":"516d62a3-1c72-4f8c-8e36-85b0ad205065","Type":"ContainerStarted","Data":"45a1ba4a7cc7a1b26665d3a9abbeee9b9578fdab932054fcf3476997647c81df"} Jul 10 22:42:24.043360 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.043330 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rx6mq" event={"ID":"0f40b374-3519-464b-b29e-c3c2303f7fa8","Type":"ContainerStarted","Data":"cead4d0ec5ec049f0cf47b2ade7a627a203665497a9393f8197294db9b3e67f9"} Jul 10 22:42:24.044748 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.044723 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"2fb68680fafd60eb02fd58a6dc749830e3f5a40b320f90e32c6e042719c21ef1"} Jul 10 22:42:24.046245 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.046195 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" event={"ID":"76f2493457929e09a73d5d44d486b48d","Type":"ContainerStarted","Data":"f4d2573647bab874888daf1f8fa3ba4d6850d7d0160ed9f3962ffc0726c99b96"} Jul 10 22:42:24.048635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.048585 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sc978" event={"ID":"52f8726e-85dd-468b-9fa3-df73264a74d9","Type":"ContainerStarted","Data":"602b74ea690b2ae01a7ab3f06ffb1914477ecd427d6daca8fe8e063c36bf5c16"} Jul 10 22:42:24.050069 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.050044 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fggm7" event={"ID":"0d9861af-65ee-4e7b-94d6-1c44887a2d68","Type":"ContainerStarted","Data":"9c89d481ba249079cda9f4bd7738a0cff52864cd022831006b7458434d6c53bb"} Jul 10 22:42:24.051362 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.051331 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wm5gq" event={"ID":"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7","Type":"ContainerStarted","Data":"967bca1ca985211c52b47eed1f799d0f11c09ceaed2c677da9d3d311b9a62eaa"} Jul 10 22:42:24.052787 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.052762 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerStarted","Data":"3e97864b04e76b66c1f438f6b303bdcb58315b69b36dad21a9188cd8886ab115"} Jul 10 22:42:24.054176 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.054145 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" event={"ID":"a566dd78-47fc-444d-beee-60f784cf722d","Type":"ContainerStarted","Data":"67e24a0b8eabbdb32b72399caf74bed57bc205f285c02fe6e1ee910c12b7726c"} Jul 10 22:42:24.056225 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.056183 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" event={"ID":"2a475e5bcf6b6d12915427e27f6213c9","Type":"ContainerStarted","Data":"433d385bcdfd5d6d5518d3c584121b919bb291a7396d2826715886d6aff6ded8"} Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.701731 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.701799 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.701839 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.701920 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702012 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:26.701987466 +0000 UTC m=+5.339413123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702014 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702056 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:26.702045387 +0000 UTC m=+5.339471043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702101 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702123 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702135 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:24.704539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:24.702186 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:26.702169787 +0000 UTC m=+5.339595459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:24.973413 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.973282 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 22:37:22 +0000 UTC" deadline="2028-02-10 18:17:32.043498938 +0000 UTC" Jul 10 22:42:24.973413 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:24.973347 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13915h35m7.070155474s" Jul 10 22:42:25.035900 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:25.035863 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:25.036103 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:25.036020 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:26.038642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:26.038608 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:26.039131 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.038751 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:26.039234 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:26.039148 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:26.039290 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.039249 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:26.718892 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:26.718849 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:26.719099 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:26.718944 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:26.719099 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:26.718991 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:26.719245 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719187 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:26.719245 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719223 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:26.719245 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719238 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:26.719403 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719301 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:30.719280258 +0000 UTC m=+9.356705917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:26.719878 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719703 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:26.719878 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719753 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:30.719738941 +0000 UTC m=+9.357164600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:26.719878 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719810 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:26.719878 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:26.719841 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:30.719830538 +0000 UTC m=+9.357256194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:27.035644 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:27.035605 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:27.035841 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:27.035786 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:28.036645 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:28.035937 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:28.036645 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:28.036083 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:28.036645 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:28.036489 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:28.036645 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:28.036594 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:29.036379 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:29.036325 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:29.036562 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:29.036479 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:30.035979 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:30.035937 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:30.036489 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.036092 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:30.036489 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:30.035937 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:30.036624 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.036562 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:30.756261 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:30.756170 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:30.756261 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:30.756252 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:30.756298 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756343 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756416 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:38.756396725 +0000 UTC m=+17.393822397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756430 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756434 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756461 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756476 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756481 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:38.75646593 +0000 UTC m=+17.393891588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:30.756549 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:30.756532 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:38.756517974 +0000 UTC m=+17.393943633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:31.036285 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:31.036185 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:31.036775 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:31.036365 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:32.036539 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:32.036350 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:32.036539 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:32.036471 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:32.037096 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:32.036879 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:32.037096 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:32.036986 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:33.036123 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:33.035847 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:33.036329 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:33.036266 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:34.035963 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:34.035846 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:34.035963 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:34.035879 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:34.036514 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:34.036001 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:34.036514 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:34.036097 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:35.036193 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:35.036158 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:35.036638 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:35.036321 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:36.035794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:36.035765 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:36.036006 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:36.035927 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:36.036006 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:36.035977 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:36.036129 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:36.036106 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:37.036176 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:37.036137 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:37.036613 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:37.036296 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:38.035789 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:38.035747 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:38.035995 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.035884 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:38.035995 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:38.035963 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:38.036108 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.036057 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:38.821808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:38.821765 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:38.821808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:38.821808 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:38.821836 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.821940 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.821959 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.821984 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.821998 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.822001 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:54.821980351 +0000 UTC m=+33.459406004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.821942 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.822049 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:54.822032898 +0000 UTC m=+33.459458550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:38.822353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:38.822095 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:54.822065366 +0000 UTC m=+33.459491034 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:39.035892 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:39.035856 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:39.036105 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:39.036010 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:40.036336 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:40.036299 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:40.036816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:40.036443 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:40.036816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:40.036477 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:40.036816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:40.036607 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:41.035568 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:41.035532 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:41.035742 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:41.035673 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:42.036440 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:42.036410 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:42.036912 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:42.036559 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:42.036912 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:42.036587 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:42.036912 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:42.036703 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:43.036266 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:43.036225 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:43.036469 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:43.036366 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:44.035492 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:44.035471 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:44.035580 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:44.035471 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:44.035637 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:44.035587 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:44.035637 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:44.035630 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:45.036743 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.036382 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:45.037588 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:45.036852 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:45.160879 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.160838 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5rk7" event={"ID":"2e19610a-10bf-4e53-8a48-eb5bb7363d1a","Type":"ContainerStarted","Data":"5a8bb3f8fa899b59e4fb6ded27b8b3b93868cf7410c22427c71253a672c7a4ca"} Jul 10 22:42:45.162433 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.162404 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rx6mq" event={"ID":"0f40b374-3519-464b-b29e-c3c2303f7fa8","Type":"ContainerStarted","Data":"fe8c736b999b9ad6fd5cb668d023c2971b9ecb0a9956199cddb09c3619b72f0b"} Jul 10 22:42:45.165387 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.165354 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"3bd282528bab41e251382eda5f40448893a642fd81d428e99b53846eb6cc4d66"} Jul 10 22:42:45.165549 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.165443 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"6132a6705536a56ecf6671746a2fe44f8241dd60f2c25fc7ba1f88557417a3cf"} Jul 10 22:42:45.165549 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.165489 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"90c9e44a08fe3f670f22727a1028a7ce48908b35f059c63916d9ee573efb487f"} Jul 10 22:42:45.165549 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.165533 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"ec57a40e84cd4f0c47ab6e731cf374d8b8c8cba4d9245c48de5f56b166d616f0"} Jul 10 22:42:45.165751 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.165577 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"e7dc097994a6d1509da22726a9c2bc64f28df778da4dca941516df68a4517cee"} Jul 10 22:42:45.165751 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.165628 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"bef5b4cead9c538a4732af67549ed10fe8f5fc1e53369cd45d8fb28b571731ce"} Jul 10 22:42:45.167116 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.167091 2551 generic.go:356] "Generic (PLEG): container finished" podID="76f2493457929e09a73d5d44d486b48d" containerID="541deb396ef897969c620d88c65f351160445467798a35584cd41a3a5dc3571e" exitCode=0 Jul 10 22:42:45.167264 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.167171 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" event={"ID":"76f2493457929e09a73d5d44d486b48d","Type":"ContainerDied","Data":"541deb396ef897969c620d88c65f351160445467798a35584cd41a3a5dc3571e"} Jul 10 22:42:45.168907 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.168879 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sc978" event={"ID":"52f8726e-85dd-468b-9fa3-df73264a74d9","Type":"ContainerStarted","Data":"39792bd5244cee49fb788287bd73d30331245e4fee3d62ca98bc479b8306d050"} Jul 10 22:42:45.170639 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.170615 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fggm7" event={"ID":"0d9861af-65ee-4e7b-94d6-1c44887a2d68","Type":"ContainerStarted","Data":"d094fcb5c10fd0b0817bd5be1e2128b1565b42ac6f245d56b8de4e5568c46220"} Jul 10 22:42:45.176173 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.176142 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wm5gq" event={"ID":"098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7","Type":"ContainerStarted","Data":"94b07a9af0e8e9dcae218c4320540a973c84ac9ee4d44ae61eb91d8404a24f2d"} Jul 10 22:42:45.177197 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.177143 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l5rk7" podStartSLOduration=3.064000921 podStartE2EDuration="23.177125892s" totalImagesPullingTime="20.113124971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:45.176728436 +0000 UTC m=+23.814154111" watchObservedRunningTime="2026-07-10 22:42:45.177125892 +0000 UTC m=+23.814551565" Jul 10 22:42:45.178123 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.178046 2551 generic.go:356] "Generic (PLEG): container finished" podID="2f6886d0-03e4-4074-9a3d-6105b77c91cc" containerID="99dcc37cab501a4a561e6e06c3b36678e6084e214a2c6b1479b5f03de714cfd5" exitCode=0 Jul 10 22:42:45.178260 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.178134 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerDied","Data":"99dcc37cab501a4a561e6e06c3b36678e6084e214a2c6b1479b5f03de714cfd5"} Jul 10 22:42:45.179833 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.179808 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" event={"ID":"a566dd78-47fc-444d-beee-60f784cf722d","Type":"ContainerStarted","Data":"06fed13ebfb8857fddba72bd92922a7c5c39d8dcfbfd407277397b3bd9c77a5e"} Jul 10 22:42:45.181272 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.181252 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" event={"ID":"2a475e5bcf6b6d12915427e27f6213c9","Type":"ContainerStarted","Data":"2a9c0126f529f8824fc39cfdec3c354295e4ff78b1418abca99a0f3bcbf9bf52"} Jul 10 22:42:45.204909 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.204848 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wm5gq" podStartSLOduration=2.915701692 podStartE2EDuration="23.204830286s" totalImagesPullingTime="20.289128594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:45.190072824 +0000 UTC m=+23.827498503" watchObservedRunningTime="2026-07-10 22:42:45.204830286 +0000 UTC m=+23.842255958" Jul 10 22:42:45.220621 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.220555 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rx6mq" podStartSLOduration=2.854754852 podStartE2EDuration="23.220534212s" totalImagesPullingTime="20.36577936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:45.219909032 +0000 UTC m=+23.857334708" watchObservedRunningTime="2026-07-10 22:42:45.220534212 +0000 UTC m=+23.857959887" Jul 10 22:42:45.235819 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.235771 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fggm7" podStartSLOduration=2.90865728 podStartE2EDuration="23.235756238s" totalImagesPullingTime="20.327098958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:45.235519261 +0000 UTC m=+23.872944936" watchObservedRunningTime="2026-07-10 22:42:45.235756238 +0000 UTC m=+23.873181912" Jul 10 22:42:45.267438 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.267334 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sc978" podStartSLOduration=3.126653548 podStartE2EDuration="23.267307839s" totalImagesPullingTime="20.140654291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:45.25214901 +0000 UTC m=+23.889574696" watchObservedRunningTime="2026-07-10 22:42:45.267307839 +0000 UTC m=+23.904733514" Jul 10 22:42:45.267991 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.267947 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-188.ec2.internal" podStartSLOduration=23.26793387 podStartE2EDuration="23.26793387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:45.266728663 +0000 UTC m=+23.904154339" watchObservedRunningTime="2026-07-10 22:42:45.26793387 +0000 UTC m=+23.905359539" Jul 10 22:42:45.848691 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:45.848659 2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 22:42:46.019765 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.019624 2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T22:42:45.848681112Z","UUID":"22666182-68d9-41da-8a80-6189d77ad62e","Handler":null,"Name":"","Endpoint":""} Jul 10 22:42:46.021486 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.021460 2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 22:42:46.021639 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.021501 2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 22:42:46.035954 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.035917 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:46.036125 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.035917 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:46.036125 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:46.036099 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:46.036240 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:46.036156 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:46.185916 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.185871 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cgxzr" event={"ID":"516d62a3-1c72-4f8c-8e36-85b0ad205065","Type":"ContainerStarted","Data":"6f51245b6beb81e4c6758b6fe4e4054ac8021e8a1fda06ad80d7000e0aa72063"} Jul 10 22:42:46.188093 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.188054 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" event={"ID":"76f2493457929e09a73d5d44d486b48d","Type":"ContainerStarted","Data":"66c21b057dee49202b380de4665f4a872010c041f3568972f0926fd2957dc3cd"} Jul 10 22:42:46.193194 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.192289 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" event={"ID":"a566dd78-47fc-444d-beee-60f784cf722d","Type":"ContainerStarted","Data":"c9cf12ede53551ece16188200600522ceb8ff6190a106037100c6d5009830ce2"} Jul 10 22:42:46.204950 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:46.204883 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cgxzr" podStartSLOduration=3.86818744 podStartE2EDuration="24.204864862s" totalImagesPullingTime="20.336677422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:46.2042743 +0000 UTC m=+24.841699975" watchObservedRunningTime="2026-07-10 22:42:46.204864862 +0000 UTC m=+24.842290536" Jul 10 22:42:47.036090 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:47.035982 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:47.036308 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:47.036131 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:47.197558 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:47.197450 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" event={"ID":"a566dd78-47fc-444d-beee-60f784cf722d","Type":"ContainerStarted","Data":"f31ae7c53e19da7fccf3b5e5d29cc528b262f08d89637c083e2f40ccc8f21b09"} Jul 10 22:42:47.201599 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:47.201564 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"40daa166bcc7910a2c695164e017a5960816edbfe25b785c219349571d2d857f"} Jul 10 22:42:47.214713 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:47.214655 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-188.ec2.internal" podStartSLOduration=25.214636824 podStartE2EDuration="25.214636824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:46.218877779 +0000 UTC m=+24.856303461" watchObservedRunningTime="2026-07-10 22:42:47.214636824 +0000 UTC m=+25.852062499" Jul 10 22:42:47.214993 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:47.214962 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4j6q9" podStartSLOduration=2.504168833 podStartE2EDuration="25.214953354s" totalImagesPullingTime="22.710784521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:47.214395345 +0000 UTC m=+25.851821021" watchObservedRunningTime="2026-07-10 22:42:47.214953354 +0000 UTC m=+25.852379028" Jul 10 22:42:48.035813 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:48.035773 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:48.036024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:48.035825 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:48.036024 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:48.035913 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:48.036145 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:48.036027 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:48.891888 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:48.891582 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:48.892448 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:48.892377 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:49.036419 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:49.036383 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:49.036622 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:49.036549 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:49.206964 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:49.206886 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:49.207421 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:49.207397 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l5rk7" Jul 10 22:42:50.036375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.036144 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:50.037068 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.036144 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:50.037068 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:50.036459 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:50.037068 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:50.036539 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:50.213227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.213071 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" event={"ID":"fcf08e02-1e2e-470b-be75-c145cdd82140","Type":"ContainerStarted","Data":"ddc0b7de9a1ce9220c5488964ab66110c539f8596aa80f65d9525ced4b2e2284"} Jul 10 22:42:50.213440 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.213417 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:50.213517 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.213500 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:50.215143 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.215112 2551 generic.go:356] "Generic (PLEG): container finished" podID="2f6886d0-03e4-4074-9a3d-6105b77c91cc" containerID="bb521915f81e6bccff3e4fb29b23d215395a63058ce8c2324be31c18b2f66219" exitCode=0 Jul 10 22:42:50.215351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.215194 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerDied","Data":"bb521915f81e6bccff3e4fb29b23d215395a63058ce8c2324be31c18b2f66219"} Jul 10 22:42:50.230480 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.230454 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:50.239084 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:50.238224 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" podStartSLOduration=7.778094625 podStartE2EDuration="28.238189098s" totalImagesPullingTime="20.460094473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:42:50.236684413 +0000 UTC m=+28.874110088" watchObservedRunningTime="2026-07-10 22:42:50.238189098 +0000 UTC m=+28.875614773" Jul 10 22:42:51.036298 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.036251 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:51.036754 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:51.036409 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:51.217911 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.217864 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:51.236089 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.236057 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:42:51.711396 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.711311 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jxrxm"] Jul 10 22:42:51.711556 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.711442 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:51.711556 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:51.711530 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:51.714593 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.714558 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-28nzx"] Jul 10 22:42:51.714768 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.714707 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:51.714853 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:51.714829 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:51.715245 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.715202 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qgkfr"] Jul 10 22:42:51.715380 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:51.715329 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:51.715450 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:51.715433 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:52.222067 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:52.222031 2551 generic.go:356] "Generic (PLEG): container finished" podID="2f6886d0-03e4-4074-9a3d-6105b77c91cc" containerID="39bd149c7cb1990b0298f286cf0317ac93c49a37540a7219600bde07aa7e85e7" exitCode=0 Jul 10 22:42:52.222567 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:52.222121 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerDied","Data":"39bd149c7cb1990b0298f286cf0317ac93c49a37540a7219600bde07aa7e85e7"} Jul 10 22:42:53.036032 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:53.035988 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:53.036032 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:53.036014 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:53.036032 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:53.036032 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:53.036329 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:53.036141 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:53.036329 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:53.036205 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:53.036329 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:53.036298 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:54.229878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:54.229765 2551 generic.go:356] "Generic (PLEG): container finished" podID="2f6886d0-03e4-4074-9a3d-6105b77c91cc" containerID="9f0a94c6f880fb56c2f1b5e2b8e2cb3e2665700ed0824367bfe93e9b7b889ec4" exitCode=0 Jul 10 22:42:54.229878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:54.229848 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerDied","Data":"9f0a94c6f880fb56c2f1b5e2b8e2cb3e2665700ed0824367bfe93e9b7b889ec4"} Jul 10 22:42:54.834180 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:54.834140 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:54.834180 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:54.834185 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:54.834205 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834333 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834358 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834388 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs podName:931fa3c6-353f-43b5-9016-bfae335b8cc7 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:26.834371774 +0000 UTC m=+65.471797427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs") pod "network-metrics-daemon-28nzx" (UID: "931fa3c6-353f-43b5-9016-bfae335b8cc7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834388 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834405 2551 projected.go:196] Error preparing data for projected volume kube-api-access-4jz5w for pod openshift-network-diagnostics/network-check-target-jxrxm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834333 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834458 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w podName:736795d7-52c1-4a00-98c9-6b0caf6eb661 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:26.834437697 +0000 UTC m=+65.471863350 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jz5w" (UniqueName: "kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w") pod "network-check-target-jxrxm" (UID: "736795d7-52c1-4a00-98c9-6b0caf6eb661") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 22:42:54.834488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:54.834479 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret podName:60803b6d-b5c5-4c66-b602-f4e7c229ce13 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:26.83446881 +0000 UTC m=+65.471894466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret") pod "global-pull-secret-syncer-qgkfr" (UID: "60803b6d-b5c5-4c66-b602-f4e7c229ce13") : object "kube-system"/"original-pull-secret" not registered Jul 10 22:42:55.035658 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:55.035611 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:55.035853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:55.035670 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:55.035853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:55.035673 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:55.035853 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:55.035759 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:55.035981 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:55.035870 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:55.035981 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:55.035946 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:57.035830 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:57.035631 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:57.036353 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:57.035631 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:57.036353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:57.035905 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qgkfr" podUID="60803b6d-b5c5-4c66-b602-f4e7c229ce13" Jul 10 22:42:57.036353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:57.036004 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxrxm" podUID="736795d7-52c1-4a00-98c9-6b0caf6eb661" Jul 10 22:42:57.036353 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:57.035631 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:57.036353 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:57.036114 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28nzx" podUID="931fa3c6-353f-43b5-9016-bfae335b8cc7" Jul 10 22:42:58.110416 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.110388 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-188.ec2.internal" event="NodeReady" Jul 10 22:42:58.110881 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.110528 2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 22:42:58.150023 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.149982 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw"] Jul 10 22:42:58.154101 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.154065 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-p7ldl"] Jul 10 22:42:58.154999 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.154516 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.155962 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.155940 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm"] Jul 10 22:42:58.156120 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.156100 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.157121 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.157095 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 22:42:58.157613 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.157582 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 22:42:58.158772 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.157850 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4"] Jul 10 22:42:58.158772 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.157964 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" Jul 10 22:42:58.158772 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.158581 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.158772 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.158592 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.158772 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.158630 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nwrhw\"" Jul 10 22:42:58.159054 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.158895 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 22:42:58.159662 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.159642 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb"] Jul 10 22:42:58.159775 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.159752 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.160279 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.160260 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.162174 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.161734 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.162174 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.161953 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 22:42:58.162414 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.162392 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-742zm\"" Jul 10 22:42:58.162639 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.162616 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b6b58f5dd-wvj84"] Jul 10 22:42:58.162809 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.162440 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.166243 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.165150 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.166243 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.165685 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wgf5s\"" Jul 10 22:42:58.166808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.166741 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.167540 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.167514 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 22:42:58.167655 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.167641 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.167797 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.167779 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.169717 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.167797 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx"] Jul 10 22:42:58.169717 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.167876 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.169717 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.168204 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hxj4g\"" Jul 10 22:42:58.170764 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.170741 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b57bc954d-h2spw"] Jul 10 22:42:58.171871 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.171846 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 22:42:58.172182 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.171883 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.172673 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.172651 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-787445c87-fv27n"] Jul 10 22:42:58.172763 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.172707 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 22:42:58.173089 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.173071 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.173908 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.173888 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 22:42:58.174045 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.174031 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zw2wb\"" Jul 10 22:42:58.174252 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.174235 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 22:42:58.174427 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.174411 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 22:42:58.174515 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.174414 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q"] Jul 10 22:42:58.174599 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.174547 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.175266 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.174503 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.175360 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.175275 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.175454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.175439 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 22:42:58.176122 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.175794 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n52zx\"" Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.181955 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-kk4fj"] Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.182892 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183315 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183427 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183653 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183742 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.183794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183774 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 22:42:58.184231 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183828 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" Jul 10 22:42:58.184231 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183899 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kmgqv\"" Jul 10 22:42:58.184231 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.183933 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 22:42:58.184231 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184071 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 22:42:58.184417 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184316 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 22:42:58.184417 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184389 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6t25n\"" Jul 10 22:42:58.184563 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184413 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 22:42:58.184670 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184648 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.184751 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184717 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw"] Jul 10 22:42:58.184751 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184754 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm"] Jul 10 22:42:58.184935 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184777 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4n7g4"] Jul 10 22:42:58.184935 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.184915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.188508 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.188476 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.188916 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.188898 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.189372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.189348 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7ds5q\"" Jul 10 22:42:58.189774 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.189755 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 22:42:58.189881 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.189795 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 22:42:58.190128 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.190109 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.190205 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.190174 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-sljdn\"" Jul 10 22:42:58.190457 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.190110 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.190748 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.190722 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx"] Jul 10 22:42:58.190895 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.190884 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dj8lx"] Jul 10 22:42:58.192130 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.192101 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.195901 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.195877 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.196771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.195774 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-p7ldl"] Jul 10 22:42:58.196771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.196266 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4"] Jul 10 22:42:58.196771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.196313 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b57bc954d-h2spw"] Jul 10 22:42:58.196771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.196337 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb"] Jul 10 22:42:58.196771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.196363 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q"] Jul 10 22:42:58.197093 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.196784 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 22:42:58.197093 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.196814 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j4q74\"" Jul 10 22:42:58.198883 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.197479 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 22:42:58.198883 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.197879 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kk4fj"] Jul 10 22:42:58.199876 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.199654 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 22:42:58.200105 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.200084 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-d7gcw\"" Jul 10 22:42:58.200860 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.200842 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 22:42:58.201172 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.200857 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 22:42:58.203085 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.203063 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 22:42:58.208464 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.208269 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dj8lx"] Jul 10 22:42:58.209937 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.209170 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6b58f5dd-wvj84"] Jul 10 22:42:58.210878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.210852 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787445c87-fv27n"] Jul 10 22:42:58.211764 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.211741 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4n7g4"] Jul 10 22:42:58.259508 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259473 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.259632 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259519 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db6471ad-9a09-4ecc-8af9-8a0659846986-serving-cert\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.259632 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259548 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61e7fa0b-2a53-430b-ae38-a7637a40a384-serving-cert\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.259632 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259571 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-ca-trust-extracted\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.259632 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259595 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c729d5-a628-44ee-aaff-979c8a614328-trusted-ca\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.259632 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259619 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4mpg\" (UniqueName: \"kubernetes.io/projected/09c729d5-a628-44ee-aaff-979c8a614328-kube-api-access-g4mpg\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.259804 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259647 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btlqg\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-kube-api-access-btlqg\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.259804 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259674 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf7dg\" (UniqueName: \"kubernetes.io/projected/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-kube-api-access-lf7dg\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.259804 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259699 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db6471ad-9a09-4ecc-8af9-8a0659846986-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.259804 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259731 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61e7fa0b-2a53-430b-ae38-a7637a40a384-config\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.259948 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259806 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a54c39d3-2d7e-4795-8043-c13457d5f0e8-ca-trust-extracted\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.259948 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259860 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-trusted-ca\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.259948 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259891 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.259948 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259922 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9cx4\" (UniqueName: \"kubernetes.io/projected/61e7fa0b-2a53-430b-ae38-a7637a40a384-kube-api-access-r9cx4\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.260134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.259948 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-installation-pull-secrets\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.260134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260000 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-stats-auth\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.260134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260053 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db6471ad-9a09-4ecc-8af9-8a0659846986-tmp\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.260134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260079 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phb2b\" (UniqueName: \"kubernetes.io/projected/db6471ad-9a09-4ecc-8af9-8a0659846986-kube-api-access-phb2b\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.260134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260106 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbwnk\" (UniqueName: \"kubernetes.io/projected/2b6daf41-2288-4cbf-ac15-2e333b6870ce-kube-api-access-vbwnk\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260140 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6hhq\" (UniqueName: \"kubernetes.io/projected/6e04e702-7813-4ab9-81f2-41d407e0afe8-kube-api-access-g6hhq\") pod \"volume-data-source-validator-78495597f6-bdjkm\" (UID: \"6e04e702-7813-4ab9-81f2-41d407e0afe8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260189 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-trusted-ca\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260250 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b76215f8-9338-4d83-a9e2-62779b4f100f-cert\") pod \"ingress-canary-dj8lx\" (UID: \"b76215f8-9338-4d83-a9e2-62779b4f100f\") " pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260280 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260303 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354a05bd-82c0-4502-8e66-468ad5da070a-tmp-dir\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260330 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gqwm\" (UniqueName: \"kubernetes.io/projected/9e113a39-f272-431f-a6e5-818d5a373fc8-kube-api-access-2gqwm\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260359 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wd8r\" (UniqueName: \"kubernetes.io/projected/354a05bd-82c0-4502-8e66-468ad5da070a-kube-api-access-7wd8r\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.260388 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260388 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrqcc\" (UniqueName: \"kubernetes.io/projected/b76215f8-9338-4d83-a9e2-62779b4f100f-kube-api-access-mrqcc\") pod \"ingress-canary-dj8lx\" (UID: \"b76215f8-9338-4d83-a9e2-62779b4f100f\") " pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260416 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jjvl\" (UniqueName: \"kubernetes.io/projected/20ba6959-4ccb-41fc-be50-60195e103f97-kube-api-access-7jjvl\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260462 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-bound-sa-token\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260494 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260528 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db6471ad-9a09-4ecc-8af9-8a0659846986-snapshots\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260570 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-default-certificate\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260625 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-bound-sa-token\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.260752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260654 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-installation-pull-secrets\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260767 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-image-registry-private-configuration\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260824 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kztg2\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-kube-api-access-kztg2\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260856 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260883 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260910 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b6daf41-2288-4cbf-ac15-2e333b6870ce-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260939 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b6daf41-2288-4cbf-ac15-2e333b6870ce-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260962 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c729d5-a628-44ee-aaff-979c8a614328-config\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.260985 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsh96\" (UniqueName: \"kubernetes.io/projected/f08bbc7b-4f89-4159-862c-549e4052fe0a-kube-api-access-fsh96\") pod \"network-check-source-869dcfcdcd-xjs6q\" (UID: \"f08bbc7b-4f89-4159-862c-549e4052fe0a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261006 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c729d5-a628-44ee-aaff-979c8a614328-serving-cert\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261032 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db6471ad-9a09-4ecc-8af9-8a0659846986-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.261145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261097 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-certificates\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.261654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261149 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.261654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261182 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-image-registry-private-configuration\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.261654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261224 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.261654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261273 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-certificates\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.261654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.261355 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/354a05bd-82c0-4502-8e66-468ad5da070a-config-volume\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.361956 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.361863 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-certificates\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.361956 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.361909 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.361956 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.361937 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-image-registry-private-configuration\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.361962 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.362030 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.362053 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787445c87-fv27n: secret "image-registry-tls" not found Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362058 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-certificates\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.362109 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls podName:4f5403e3-e48e-4ec1-9d37-9553b9689ca5 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.862088075 +0000 UTC m=+37.499513733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls") pod "image-registry-787445c87-fv27n" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5") : secret "image-registry-tls" not found Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362130 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/354a05bd-82c0-4502-8e66-468ad5da070a-config-volume\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.362147 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.862128741 +0000 UTC m=+37.499554417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : configmap references non-existent config key: service-ca.crt Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362182 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362227 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db6471ad-9a09-4ecc-8af9-8a0659846986-serving-cert\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362252 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61e7fa0b-2a53-430b-ae38-a7637a40a384-serving-cert\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362275 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-ca-trust-extracted\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.362289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362299 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c729d5-a628-44ee-aaff-979c8a614328-trusted-ca\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362322 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4mpg\" (UniqueName: \"kubernetes.io/projected/09c729d5-a628-44ee-aaff-979c8a614328-kube-api-access-g4mpg\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362354 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btlqg\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-kube-api-access-btlqg\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362382 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf7dg\" (UniqueName: \"kubernetes.io/projected/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-kube-api-access-lf7dg\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362406 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db6471ad-9a09-4ecc-8af9-8a0659846986-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362442 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61e7fa0b-2a53-430b-ae38-a7637a40a384-config\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362471 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a54c39d3-2d7e-4795-8043-c13457d5f0e8-ca-trust-extracted\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362496 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-trusted-ca\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362520 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362548 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9cx4\" (UniqueName: \"kubernetes.io/projected/61e7fa0b-2a53-430b-ae38-a7637a40a384-kube-api-access-r9cx4\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362576 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-installation-pull-secrets\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362598 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-stats-auth\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362633 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db6471ad-9a09-4ecc-8af9-8a0659846986-tmp\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362641 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-certificates\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362657 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phb2b\" (UniqueName: \"kubernetes.io/projected/db6471ad-9a09-4ecc-8af9-8a0659846986-kube-api-access-phb2b\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362681 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/354a05bd-82c0-4502-8e66-468ad5da070a-config-volume\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.362868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362692 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-certificates\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362697 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbwnk\" (UniqueName: \"kubernetes.io/projected/2b6daf41-2288-4cbf-ac15-2e333b6870ce-kube-api-access-vbwnk\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362730 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6hhq\" (UniqueName: \"kubernetes.io/projected/6e04e702-7813-4ab9-81f2-41d407e0afe8-kube-api-access-g6hhq\") pod \"volume-data-source-validator-78495597f6-bdjkm\" (UID: \"6e04e702-7813-4ab9-81f2-41d407e0afe8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362758 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-trusted-ca\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362781 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b76215f8-9338-4d83-a9e2-62779b4f100f-cert\") pod \"ingress-canary-dj8lx\" (UID: \"b76215f8-9338-4d83-a9e2-62779b4f100f\") " pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362807 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362832 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354a05bd-82c0-4502-8e66-468ad5da070a-tmp-dir\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362860 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gqwm\" (UniqueName: \"kubernetes.io/projected/9e113a39-f272-431f-a6e5-818d5a373fc8-kube-api-access-2gqwm\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362889 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wd8r\" (UniqueName: \"kubernetes.io/projected/354a05bd-82c0-4502-8e66-468ad5da070a-kube-api-access-7wd8r\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362920 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrqcc\" (UniqueName: \"kubernetes.io/projected/b76215f8-9338-4d83-a9e2-62779b4f100f-kube-api-access-mrqcc\") pod \"ingress-canary-dj8lx\" (UID: \"b76215f8-9338-4d83-a9e2-62779b4f100f\") " pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362945 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jjvl\" (UniqueName: \"kubernetes.io/projected/20ba6959-4ccb-41fc-be50-60195e103f97-kube-api-access-7jjvl\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.362972 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-bound-sa-token\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363007 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363037 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db6471ad-9a09-4ecc-8af9-8a0659846986-snapshots\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363062 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-default-certificate\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363097 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-bound-sa-token\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.363683 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363124 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-installation-pull-secrets\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363153 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-image-registry-private-configuration\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363179 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kztg2\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-kube-api-access-kztg2\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363231 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363237 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a54c39d3-2d7e-4795-8043-c13457d5f0e8-ca-trust-extracted\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363262 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363293 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b6daf41-2288-4cbf-ac15-2e333b6870ce-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363323 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b6daf41-2288-4cbf-ac15-2e333b6870ce-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363356 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c729d5-a628-44ee-aaff-979c8a614328-config\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363384 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsh96\" (UniqueName: \"kubernetes.io/projected/f08bbc7b-4f89-4159-862c-549e4052fe0a-kube-api-access-fsh96\") pod \"network-check-source-869dcfcdcd-xjs6q\" (UID: \"f08bbc7b-4f89-4159-862c-549e4052fe0a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363387 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61e7fa0b-2a53-430b-ae38-a7637a40a384-config\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363413 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c729d5-a628-44ee-aaff-979c8a614328-serving-cert\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363444 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db6471ad-9a09-4ecc-8af9-8a0659846986-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363455 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.363576 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.363644 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls podName:354a05bd-82c0-4502-8e66-468ad5da070a nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.863621231 +0000 UTC m=+37.501046898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls") pod "dns-default-4n7g4" (UID: "354a05bd-82c0-4502-8e66-468ad5da070a") : secret "dns-default-metrics-tls" not found Jul 10 22:42:58.364454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.363882 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354a05bd-82c0-4502-8e66-468ad5da070a-tmp-dir\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.364052 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-trusted-ca\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.364204 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db6471ad-9a09-4ecc-8af9-8a0659846986-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.364653 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.364821 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.364883 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls podName:20ba6959-4ccb-41fc-be50-60195e103f97 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.864852911 +0000 UTC m=+37.502278569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls") pod "cluster-samples-operator-65db858894-74wz4" (UID: "20ba6959-4ccb-41fc-be50-60195e103f97") : secret "samples-operator-tls" not found Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.365001 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.864982368 +0000 UTC m=+37.502408034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : secret "router-metrics-certs-default" not found Jul 10 22:42:58.365324 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.365334 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db6471ad-9a09-4ecc-8af9-8a0659846986-tmp\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.365827 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.365608 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db6471ad-9a09-4ecc-8af9-8a0659846986-snapshots\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.365827 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.365682 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-ca-trust-extracted\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.365827 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.365721 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c729d5-a628-44ee-aaff-979c8a614328-config\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.365827 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.365780 2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 22:42:58.366070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.365867 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls podName:90b8a8ae-1d63-4e73-8d39-2821e3ec325b nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.865845272 +0000 UTC m=+37.503270926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p5drw" (UID: "90b8a8ae-1d63-4e73-8d39-2821e3ec325b") : secret "cluster-monitoring-operator-tls" not found Jul 10 22:42:58.366070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.365954 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:42:58.366070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.365971 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b58f5dd-wvj84: secret "image-registry-tls" not found Jul 10 22:42:58.366070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.366018 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls podName:a54c39d3-2d7e-4795-8043-c13457d5f0e8 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:58.866002048 +0000 UTC m=+37.503427704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls") pod "image-registry-6b6b58f5dd-wvj84" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8") : secret "image-registry-tls" not found Jul 10 22:42:58.366757 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.366725 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b6daf41-2288-4cbf-ac15-2e333b6870ce-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.367105 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.367080 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c729d5-a628-44ee-aaff-979c8a614328-trusted-ca\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.370057 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.370025 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db6471ad-9a09-4ecc-8af9-8a0659846986-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.370178 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.370147 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-image-registry-private-configuration\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.370651 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.370622 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c729d5-a628-44ee-aaff-979c8a614328-serving-cert\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.371011 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.370991 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61e7fa0b-2a53-430b-ae38-a7637a40a384-serving-cert\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.371440 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-installation-pull-secrets\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.371683 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-trusted-ca\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.371843 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-installation-pull-secrets\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.371866 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b6daf41-2288-4cbf-ac15-2e333b6870ce-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.372067 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-image-registry-private-configuration\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.372202 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b76215f8-9338-4d83-a9e2-62779b4f100f-cert\") pod \"ingress-canary-dj8lx\" (UID: \"b76215f8-9338-4d83-a9e2-62779b4f100f\") " pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.372664 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-stats-auth\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.373076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.372922 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-default-certificate\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.379262 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.378582 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db6471ad-9a09-4ecc-8af9-8a0659846986-serving-cert\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.379427 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.379261 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf7dg\" (UniqueName: \"kubernetes.io/projected/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-kube-api-access-lf7dg\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.382263 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wd8r\" (UniqueName: \"kubernetes.io/projected/354a05bd-82c0-4502-8e66-468ad5da070a-kube-api-access-7wd8r\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.382519 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsh96\" (UniqueName: \"kubernetes.io/projected/f08bbc7b-4f89-4159-862c-549e4052fe0a-kube-api-access-fsh96\") pod \"network-check-source-869dcfcdcd-xjs6q\" (UID: \"f08bbc7b-4f89-4159-862c-549e4052fe0a\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.382769 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phb2b\" (UniqueName: \"kubernetes.io/projected/db6471ad-9a09-4ecc-8af9-8a0659846986-kube-api-access-phb2b\") pod \"insights-operator-68d9dd94cf-p7ldl\" (UID: \"db6471ad-9a09-4ecc-8af9-8a0659846986\") " pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.383439 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btlqg\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-kube-api-access-btlqg\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.384012 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jjvl\" (UniqueName: \"kubernetes.io/projected/20ba6959-4ccb-41fc-be50-60195e103f97-kube-api-access-7jjvl\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.384397 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-bound-sa-token\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.384988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.384830 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9cx4\" (UniqueName: \"kubernetes.io/projected/61e7fa0b-2a53-430b-ae38-a7637a40a384-kube-api-access-r9cx4\") pod \"service-ca-operator-7695fc6c8c-gprfb\" (UID: \"61e7fa0b-2a53-430b-ae38-a7637a40a384\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.388111 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.388075 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbwnk\" (UniqueName: \"kubernetes.io/projected/2b6daf41-2288-4cbf-ac15-2e333b6870ce-kube-api-access-vbwnk\") pod \"kube-storage-version-migrator-operator-799cb7f958-lg7dx\" (UID: \"2b6daf41-2288-4cbf-ac15-2e333b6870ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.388797 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.388767 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrqcc\" (UniqueName: \"kubernetes.io/projected/b76215f8-9338-4d83-a9e2-62779b4f100f-kube-api-access-mrqcc\") pod \"ingress-canary-dj8lx\" (UID: \"b76215f8-9338-4d83-a9e2-62779b4f100f\") " pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.388901 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.388821 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-bound-sa-token\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.390045 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.389445 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gqwm\" (UniqueName: \"kubernetes.io/projected/9e113a39-f272-431f-a6e5-818d5a373fc8-kube-api-access-2gqwm\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.390045 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.389612 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kztg2\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-kube-api-access-kztg2\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.390045 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.389686 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4mpg\" (UniqueName: \"kubernetes.io/projected/09c729d5-a628-44ee-aaff-979c8a614328-kube-api-access-g4mpg\") pod \"console-operator-7866497494-kk4fj\" (UID: \"09c729d5-a628-44ee-aaff-979c8a614328\") " pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.390906 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.390885 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6hhq\" (UniqueName: \"kubernetes.io/projected/6e04e702-7813-4ab9-81f2-41d407e0afe8-kube-api-access-g6hhq\") pod \"volume-data-source-validator-78495597f6-bdjkm\" (UID: \"6e04e702-7813-4ab9-81f2-41d407e0afe8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" Jul 10 22:42:58.487760 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.487716 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" Jul 10 22:42:58.502882 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.502844 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" Jul 10 22:42:58.521019 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.520968 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" Jul 10 22:42:58.537349 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.537310 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" Jul 10 22:42:58.561626 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.561592 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" Jul 10 22:42:58.569669 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.569630 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:42:58.583638 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.583601 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dj8lx" Jul 10 22:42:58.870351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870312 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:58.870351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870355 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:58.870618 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870476 2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 22:42:58.870618 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870480 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:42:58.870618 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870565 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b58f5dd-wvj84: secret "image-registry-tls" not found Jul 10 22:42:58.870618 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870542 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls podName:90b8a8ae-1d63-4e73-8d39-2821e3ec325b nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.870527947 +0000 UTC m=+38.507953603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p5drw" (UID: "90b8a8ae-1d63-4e73-8d39-2821e3ec325b") : secret "cluster-monitoring-operator-tls" not found Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870625 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870665 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870693 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870703 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787445c87-fv27n: secret "image-registry-tls" not found Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870719 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls podName:a54c39d3-2d7e-4795-8043-c13457d5f0e8 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.870693512 +0000 UTC m=+38.508119178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls") pod "image-registry-6b6b58f5dd-wvj84" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8") : secret "image-registry-tls" not found Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870760 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls podName:4f5403e3-e48e-4ec1-9d37-9553b9689ca5 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.870739309 +0000 UTC m=+38.508164971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls") pod "image-registry-787445c87-fv27n" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5") : secret "image-registry-tls" not found Jul 10 22:42:58.870816 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870788 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.87077472 +0000 UTC m=+38.508200376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : configmap references non-existent config key: service-ca.crt Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870832 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870892 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870910 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:58.870938 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.870961 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.870950965 +0000 UTC m=+38.508376619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : secret "router-metrics-certs-default" not found Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.871023 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.871030 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.871075 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls podName:354a05bd-82c0-4502-8e66-468ad5da070a nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.871060141 +0000 UTC m=+38.508485807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls") pod "dns-default-4n7g4" (UID: "354a05bd-82c0-4502-8e66-468ad5da070a") : secret "dns-default-metrics-tls" not found Jul 10 22:42:58.871126 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:58.871094 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls podName:20ba6959-4ccb-41fc-be50-60195e103f97 nodeName:}" failed. No retries permitted until 2026-07-10 22:42:59.871083778 +0000 UTC m=+38.508509456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls") pod "cluster-samples-operator-65db858894-74wz4" (UID: "20ba6959-4ccb-41fc-be50-60195e103f97") : secret "samples-operator-tls" not found Jul 10 22:42:59.036929 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.036021 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:42:59.036929 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.036045 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:42:59.036929 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.036781 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:42:59.044134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.043540 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wwzq4\"" Jul 10 22:42:59.044134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.043805 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 22:42:59.044134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.043977 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 22:42:59.044788 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.044587 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vx2kj\"" Jul 10 22:42:59.282714 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.282680 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-p7ldl"] Jul 10 22:42:59.287786 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:59.287753 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6471ad_9a09_4ecc_8af9_8a0659846986.slice/crio-9892a6671d047db027766c1446d4f4706c2d7f819a883406bb5dec6c95d8cb76 WatchSource:0}: Error finding container 9892a6671d047db027766c1446d4f4706c2d7f819a883406bb5dec6c95d8cb76: Status 404 returned error can't find the container with id 9892a6671d047db027766c1446d4f4706c2d7f819a883406bb5dec6c95d8cb76 Jul 10 22:42:59.288078 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.287997 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm"] Jul 10 22:42:59.294145 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.293795 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dj8lx"] Jul 10 22:42:59.304015 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.303989 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx"] Jul 10 22:42:59.307752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.307723 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kk4fj"] Jul 10 22:42:59.311387 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:59.311360 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c729d5_a628_44ee_aaff_979c8a614328.slice/crio-07772cf14dbd6589e17dc8114a26d84e7bb2c90d83eb4d062a746cc9031026ef WatchSource:0}: Error finding container 07772cf14dbd6589e17dc8114a26d84e7bb2c90d83eb4d062a746cc9031026ef: Status 404 returned error can't find the container with id 07772cf14dbd6589e17dc8114a26d84e7bb2c90d83eb4d062a746cc9031026ef Jul 10 22:42:59.319113 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.319086 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q"] Jul 10 22:42:59.322756 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.322728 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb"] Jul 10 22:42:59.325932 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:59.325904 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf08bbc7b_4f89_4159_862c_549e4052fe0a.slice/crio-3990ebbc9ab92e3ea6dc4bd43ebb7e87970637a80a5f6f6516c0946d760f3863 WatchSource:0}: Error finding container 3990ebbc9ab92e3ea6dc4bd43ebb7e87970637a80a5f6f6516c0946d760f3863: Status 404 returned error can't find the container with id 3990ebbc9ab92e3ea6dc4bd43ebb7e87970637a80a5f6f6516c0946d760f3863 Jul 10 22:42:59.326914 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:42:59.326891 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e7fa0b_2a53_430b_ae38_a7637a40a384.slice/crio-ac0844feebec95ed4e8d421ffe5997cbe601c57f070f998ebe333392772f9415 WatchSource:0}: Error finding container ac0844feebec95ed4e8d421ffe5997cbe601c57f070f998ebe333392772f9415: Status 404 returned error can't find the container with id ac0844feebec95ed4e8d421ffe5997cbe601c57f070f998ebe333392772f9415 Jul 10 22:42:59.896836 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.896713 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:42:59.897051 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.896989 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:42:59.897171 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.897152 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:42:59.897257 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.897238 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:42:59.897433 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.897361 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:42:59.897433 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.897412 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:59.897606 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.897559 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:42:59.897851 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.897790 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 22:42:59.898070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.897933 2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 22:42:59.898070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.897992 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.897915223 +0000 UTC m=+40.535340879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : secret "router-metrics-certs-default" not found Jul 10 22:42:59.898070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898043 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 22:42:59.898070 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898050 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls podName:90b8a8ae-1d63-4e73-8d39-2821e3ec325b nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.898037881 +0000 UTC m=+40.535463539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p5drw" (UID: "90b8a8ae-1d63-4e73-8d39-2821e3ec325b") : secret "cluster-monitoring-operator-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898084 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls podName:354a05bd-82c0-4502-8e66-468ad5da070a nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.898066812 +0000 UTC m=+40.535492470 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls") pod "dns-default-4n7g4" (UID: "354a05bd-82c0-4502-8e66-468ad5da070a") : secret "dns-default-metrics-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898091 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898104 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787445c87-fv27n: secret "image-registry-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898138 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls podName:4f5403e3-e48e-4ec1-9d37-9553b9689ca5 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.898128199 +0000 UTC m=+40.535553896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls") pod "image-registry-787445c87-fv27n" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5") : secret "image-registry-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898145 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898179 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls podName:20ba6959-4ccb-41fc-be50-60195e103f97 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.898168739 +0000 UTC m=+40.535594401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls") pod "cluster-samples-operator-65db858894-74wz4" (UID: "20ba6959-4ccb-41fc-be50-60195e103f97") : secret "samples-operator-tls" not found Jul 10 22:42:59.898274 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898198 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.898185529 +0000 UTC m=+40.535611197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : configmap references non-existent config key: service-ca.crt Jul 10 22:42:59.898633 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898303 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:42:59.898633 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898317 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b58f5dd-wvj84: secret "image-registry-tls" not found Jul 10 22:42:59.898633 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:42:59.898373 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls podName:a54c39d3-2d7e-4795-8043-c13457d5f0e8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:01.898359565 +0000 UTC m=+40.535785232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls") pod "image-registry-6b6b58f5dd-wvj84" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8") : secret "image-registry-tls" not found Jul 10 22:42:59.901200 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.901100 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:42:59.983925 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:42:59.983838 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:00.042915 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.042816 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:00.255775 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.255667 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" event={"ID":"6e04e702-7813-4ab9-81f2-41d407e0afe8","Type":"ContainerStarted","Data":"c41088206bf12d28df3669e99eaebb83518ed136d072228c6e9bba631b20948d"} Jul 10 22:43:00.258275 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.258147 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" event={"ID":"db6471ad-9a09-4ecc-8af9-8a0659846986","Type":"ContainerStarted","Data":"9892a6671d047db027766c1446d4f4706c2d7f819a883406bb5dec6c95d8cb76"} Jul 10 22:43:00.278586 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.274189 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" event={"ID":"f08bbc7b-4f89-4159-862c-549e4052fe0a","Type":"ContainerStarted","Data":"3990ebbc9ab92e3ea6dc4bd43ebb7e87970637a80a5f6f6516c0946d760f3863"} Jul 10 22:43:00.289765 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.289645 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" event={"ID":"2b6daf41-2288-4cbf-ac15-2e333b6870ce","Type":"ContainerStarted","Data":"9bd5ebd116014f5756df14dfe6d7f83950626e1c7cc66f4a17729e36184c662f"} Jul 10 22:43:00.293712 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.293668 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dj8lx" event={"ID":"b76215f8-9338-4d83-a9e2-62779b4f100f","Type":"ContainerStarted","Data":"0bd593e2185d57637b546c76e4371af20235754dff2a4be109f0d5f95b0548d3"} Jul 10 22:43:00.303834 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.303796 2551 generic.go:356] "Generic (PLEG): container finished" podID="2f6886d0-03e4-4074-9a3d-6105b77c91cc" containerID="ac9b867dbed9f9b064eed5794770a5b252a088446618668c46ecba7cd3ff9ff5" exitCode=0 Jul 10 22:43:00.304195 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.303994 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerDied","Data":"ac9b867dbed9f9b064eed5794770a5b252a088446618668c46ecba7cd3ff9ff5"} Jul 10 22:43:00.308363 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.308329 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kk4fj" event={"ID":"09c729d5-a628-44ee-aaff-979c8a614328","Type":"ContainerStarted","Data":"07772cf14dbd6589e17dc8114a26d84e7bb2c90d83eb4d062a746cc9031026ef"} Jul 10 22:43:00.315045 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.314987 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" event={"ID":"61e7fa0b-2a53-430b-ae38-a7637a40a384","Type":"ContainerStarted","Data":"ac0844feebec95ed4e8d421ffe5997cbe601c57f070f998ebe333392772f9415"} Jul 10 22:43:00.956122 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:00.955925 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.084756 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.084706 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.251015 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.250830 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.326465 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.326304 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.334477 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.333345 2551 generic.go:356] "Generic (PLEG): container finished" podID="2f6886d0-03e4-4074-9a3d-6105b77c91cc" containerID="ec44ebe4f0e235083ec51f22b94f80264988ebaaf59d3f1a1186307c530f166f" exitCode=0 Jul 10 22:43:01.334477 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.333410 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerDied","Data":"ec44ebe4f0e235083ec51f22b94f80264988ebaaf59d3f1a1186307c530f166f"} Jul 10 22:43:01.391396 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.391253 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.735434 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.735375 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.820105 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.820006 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921516 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921570 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921639 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921688 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921735 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921784 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.921825 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.921990 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922005 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b58f5dd-wvj84: secret "image-registry-tls" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922066 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls podName:a54c39d3-2d7e-4795-8043-c13457d5f0e8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.922046713 +0000 UTC m=+44.559472368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls") pod "image-registry-6b6b58f5dd-wvj84" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8") : secret "image-registry-tls" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922516 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922534 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787445c87-fv27n: secret "image-registry-tls" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922584 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls podName:4f5403e3-e48e-4ec1-9d37-9553b9689ca5 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.922568563 +0000 UTC m=+44.559994222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls") pod "image-registry-787445c87-fv27n" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5") : secret "image-registry-tls" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922660 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.922648994 +0000 UTC m=+44.560074654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : configmap references non-existent config key: service-ca.crt Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922753 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922783 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.922773246 +0000 UTC m=+44.560198902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : secret "router-metrics-certs-default" not found Jul 10 22:43:01.923648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922840 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 22:43:01.924717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922869 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls podName:354a05bd-82c0-4502-8e66-468ad5da070a nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.922858857 +0000 UTC m=+44.560284517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls") pod "dns-default-4n7g4" (UID: "354a05bd-82c0-4502-8e66-468ad5da070a") : secret "dns-default-metrics-tls" not found Jul 10 22:43:01.924717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922926 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 22:43:01.924717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.922957 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls podName:20ba6959-4ccb-41fc-be50-60195e103f97 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.922946856 +0000 UTC m=+44.560372517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls") pod "cluster-samples-operator-65db858894-74wz4" (UID: "20ba6959-4ccb-41fc-be50-60195e103f97") : secret "samples-operator-tls" not found Jul 10 22:43:01.924717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.923010 2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 22:43:01.924717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:01.923039 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls podName:90b8a8ae-1d63-4e73-8d39-2821e3ec325b nodeName:}" failed. No retries permitted until 2026-07-10 22:43:05.923029319 +0000 UTC m=+44.560454973 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p5drw" (UID: "90b8a8ae-1d63-4e73-8d39-2821e3ec325b") : secret "cluster-monitoring-operator-tls" not found Jul 10 22:43:01.927898 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:01.927834 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:02.025367 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.025333 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:02.323667 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.323628 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:02.342100 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.342052 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" event={"ID":"2f6886d0-03e4-4074-9a3d-6105b77c91cc","Type":"ContainerStarted","Data":"c206d83a7d06ae64c1f38fa35f1df895b56962aba1658f82b32c825d9729f877"} Jul 10 22:43:02.344244 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.344176 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" event={"ID":"6e04e702-7813-4ab9-81f2-41d407e0afe8","Type":"ContainerStarted","Data":"63f6bbfb39616d20fab904550e4fed550eb9dd17865ea4c2957b991fd867d019"} Jul 10 22:43:02.367115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.366952 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p5d8f" podStartSLOduration=12.313496537 podStartE2EDuration="40.366928s" totalImagesPullingTime="28.053431463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:02.364870803 +0000 UTC m=+41.002296479" watchObservedRunningTime="2026-07-10 22:43:02.366928 +0000 UTC m=+41.004353677" Jul 10 22:43:02.379703 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.379638 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-bdjkm" podStartSLOduration=21.258201851 podStartE2EDuration="23.379614943s" totalImagesPullingTime="2.121413092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:02.379445178 +0000 UTC m=+41.016870898" watchObservedRunningTime="2026-07-10 22:43:02.379614943 +0000 UTC m=+41.017040619" Jul 10 22:43:02.425901 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.425857 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:02.526597 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:02.526554 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:03.112785 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:03.112744 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:03.177202 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:03.177163 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:03.525938 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:03.525900 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:03.595481 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:03.595435 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:05.964304 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964261 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964323 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964364 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964398 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964419 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964474 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964479 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964523 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964541 2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964586 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964544897 +0000 UTC m=+52.601970550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : secret "router-metrics-certs-default" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:05.964492 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964608 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964627 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787445c87-fv27n: secret "image-registry-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964613 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls podName:354a05bd-82c0-4502-8e66-468ad5da070a nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964595398 +0000 UTC m=+52.602021071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls") pod "dns-default-4n7g4" (UID: "354a05bd-82c0-4502-8e66-468ad5da070a") : secret "dns-default-metrics-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964523 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964613 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964706 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b58f5dd-wvj84: secret "image-registry-tls" not found Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964716 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964688732 +0000 UTC m=+52.602114386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : configmap references non-existent config key: service-ca.crt Jul 10 22:43:05.964810 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964737 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls podName:90b8a8ae-1d63-4e73-8d39-2821e3ec325b nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964728397 +0000 UTC m=+52.602154050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p5drw" (UID: "90b8a8ae-1d63-4e73-8d39-2821e3ec325b") : secret "cluster-monitoring-operator-tls" not found Jul 10 22:43:05.965488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964758 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls podName:4f5403e3-e48e-4ec1-9d37-9553b9689ca5 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964746111 +0000 UTC m=+52.602171765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls") pod "image-registry-787445c87-fv27n" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5") : secret "image-registry-tls" not found Jul 10 22:43:05.965488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964779 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls podName:20ba6959-4ccb-41fc-be50-60195e103f97 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964772562 +0000 UTC m=+52.602198214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls") pod "cluster-samples-operator-65db858894-74wz4" (UID: "20ba6959-4ccb-41fc-be50-60195e103f97") : secret "samples-operator-tls" not found Jul 10 22:43:05.965488 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:05.964794 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls podName:a54c39d3-2d7e-4795-8043-c13457d5f0e8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.964789614 +0000 UTC m=+52.602215267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls") pod "image-registry-6b6b58f5dd-wvj84" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8") : secret "image-registry-tls" not found Jul 10 22:43:07.363310 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.363259 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" event={"ID":"f08bbc7b-4f89-4159-862c-549e4052fe0a","Type":"ContainerStarted","Data":"eb1ff3e4d1c3b38c96d3656df1c33fd2b8f880a9b1a9a5384832450ca16304c8"} Jul 10 22:43:07.365559 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.365523 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" event={"ID":"2b6daf41-2288-4cbf-ac15-2e333b6870ce","Type":"ContainerStarted","Data":"8fa702938ba87d3a7068b6cd91d61a276da676dbf20182a05c5b5f1175bdbbdf"} Jul 10 22:43:07.367477 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.367293 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dj8lx" event={"ID":"b76215f8-9338-4d83-a9e2-62779b4f100f","Type":"ContainerStarted","Data":"a3c64096d372d990d1fb9bd75b22f6e091362b4a419527c576e43265c66751f7"} Jul 10 22:43:07.369087 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.369064 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/0.log" Jul 10 22:43:07.369247 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.369109 2551 generic.go:356] "Generic (PLEG): container finished" podID="09c729d5-a628-44ee-aaff-979c8a614328" containerID="3d6db8a5268845235121c60be013dadb4b859c0d4a68391190967dd83cc34d85" exitCode=255 Jul 10 22:43:07.369247 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.369190 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kk4fj" event={"ID":"09c729d5-a628-44ee-aaff-979c8a614328","Type":"ContainerDied","Data":"3d6db8a5268845235121c60be013dadb4b859c0d4a68391190967dd83cc34d85"} Jul 10 22:43:07.369429 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.369410 2551 scope.go:117] "RemoveContainer" containerID="3d6db8a5268845235121c60be013dadb4b859c0d4a68391190967dd83cc34d85" Jul 10 22:43:07.371671 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.371641 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" event={"ID":"61e7fa0b-2a53-430b-ae38-a7637a40a384","Type":"ContainerStarted","Data":"a66972e8ac70bb3676597ae642157151653b587b45b694e7c07fd1ad4f59e49c"} Jul 10 22:43:07.373908 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.373876 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" event={"ID":"db6471ad-9a09-4ecc-8af9-8a0659846986","Type":"ContainerStarted","Data":"87f1eeee6ed9c401cd1f7aa83171e3b832357b5e41a3aa5e58dc0985574ec1d9"} Jul 10 22:43:07.381006 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.380938 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-xjs6q" podStartSLOduration=16.640449549 podStartE2EDuration="24.380903798s" totalImagesPullingTime="7.740454249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:07.379041884 +0000 UTC m=+46.016467558" watchObservedRunningTime="2026-07-10 22:43:07.380903798 +0000 UTC m=+46.018329474" Jul 10 22:43:07.394311 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.394241 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dj8lx" podStartSLOduration=1.64099417 podStartE2EDuration="9.394204442s" totalImagesPullingTime="7.753210272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:07.393023012 +0000 UTC m=+46.030448689" watchObservedRunningTime="2026-07-10 22:43:07.394204442 +0000 UTC m=+46.031630118" Jul 10 22:43:07.411846 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.410835 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" podStartSLOduration=16.682640516 podStartE2EDuration="24.410810327s" totalImagesPullingTime="7.728169811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:07.409105549 +0000 UTC m=+46.046531225" watchObservedRunningTime="2026-07-10 22:43:07.410810327 +0000 UTC m=+46.048236005" Jul 10 22:43:07.449531 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.449462 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" podStartSLOduration=20.682105827 podStartE2EDuration="28.449438367s" totalImagesPullingTime="7.76733254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:07.448941733 +0000 UTC m=+46.086367409" watchObservedRunningTime="2026-07-10 22:43:07.449438367 +0000 UTC m=+46.086864043" Jul 10 22:43:07.487662 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:07.486844 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" podStartSLOduration=19.740452637 podStartE2EDuration="27.486820915s" totalImagesPullingTime="7.746368278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:07.485682333 +0000 UTC m=+46.123108010" watchObservedRunningTime="2026-07-10 22:43:07.486820915 +0000 UTC m=+46.124246591" Jul 10 22:43:08.220077 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.220027 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:08.380843 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.380758 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/1.log" Jul 10 22:43:08.381328 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.381310 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/0.log" Jul 10 22:43:08.381369 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.381356 2551 generic.go:356] "Generic (PLEG): container finished" podID="09c729d5-a628-44ee-aaff-979c8a614328" containerID="7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb" exitCode=255 Jul 10 22:43:08.381482 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.381460 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kk4fj" event={"ID":"09c729d5-a628-44ee-aaff-979c8a614328","Type":"ContainerDied","Data":"7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb"} Jul 10 22:43:08.381542 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.381523 2551 scope.go:117] "RemoveContainer" containerID="3d6db8a5268845235121c60be013dadb4b859c0d4a68391190967dd83cc34d85" Jul 10 22:43:08.381780 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.381754 2551 scope.go:117] "RemoveContainer" containerID="7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb" Jul 10 22:43:08.382002 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:08.381968 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kk4fj_openshift-console-operator(09c729d5-a628-44ee-aaff-979c8a614328)\"" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podUID="09c729d5-a628-44ee-aaff-979c8a614328" Jul 10 22:43:08.570303 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.570243 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:43:08.570588 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.570337 2551 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:43:08.607435 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.607401 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:08.634151 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.634061 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s"] Jul 10 22:43:08.663736 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.663686 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s"] Jul 10 22:43:08.663958 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.663843 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" Jul 10 22:43:08.666492 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.666453 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 22:43:08.666770 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.666748 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 22:43:08.667440 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.667414 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fwndd\"" Jul 10 22:43:08.687360 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.687325 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:08.738811 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.738770 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:08.807433 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.807396 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kttvz\" (UniqueName: \"kubernetes.io/projected/734bd34c-bb9b-4254-9e81-71ccce05eec8-kube-api-access-kttvz\") pod \"migrator-7f59b54f6c-5ng5s\" (UID: \"734bd34c-bb9b-4254-9e81-71ccce05eec8\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" Jul 10 22:43:08.908771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.908672 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kttvz\" (UniqueName: \"kubernetes.io/projected/734bd34c-bb9b-4254-9e81-71ccce05eec8-kube-api-access-kttvz\") pod \"migrator-7f59b54f6c-5ng5s\" (UID: \"734bd34c-bb9b-4254-9e81-71ccce05eec8\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" Jul 10 22:43:08.919468 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.919430 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kttvz\" (UniqueName: \"kubernetes.io/projected/734bd34c-bb9b-4254-9e81-71ccce05eec8-kube-api-access-kttvz\") pod \"migrator-7f59b54f6c-5ng5s\" (UID: \"734bd34c-bb9b-4254-9e81-71ccce05eec8\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" Jul 10 22:43:08.930836 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.930793 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:08.976228 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.976175 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" Jul 10 22:43:08.986387 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:08.986349 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:09.136091 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.136049 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s"] Jul 10 22:43:09.139759 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:09.139724 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734bd34c_bb9b_4254_9e81_71ccce05eec8.slice/crio-09bad50cdb664bf71384606a0af9a3f089676eb4c785317cfe018ef0779334f7 WatchSource:0}: Error finding container 09bad50cdb664bf71384606a0af9a3f089676eb4c785317cfe018ef0779334f7: Status 404 returned error can't find the container with id 09bad50cdb664bf71384606a0af9a3f089676eb4c785317cfe018ef0779334f7 Jul 10 22:43:09.386576 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.386489 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" event={"ID":"734bd34c-bb9b-4254-9e81-71ccce05eec8","Type":"ContainerStarted","Data":"09bad50cdb664bf71384606a0af9a3f089676eb4c785317cfe018ef0779334f7"} Jul 10 22:43:09.389102 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.389066 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/1.log" Jul 10 22:43:09.389674 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.389630 2551 scope.go:117] "RemoveContainer" containerID="7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb" Jul 10 22:43:09.389903 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:09.389876 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kk4fj_openshift-console-operator(09c729d5-a628-44ee-aaff-979c8a614328)\"" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podUID="09c729d5-a628-44ee-aaff-979c8a614328" Jul 10 22:43:09.421976 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.421930 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nzs4r"] Jul 10 22:43:09.443517 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.443474 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:09.453829 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.453791 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nzs4r"] Jul 10 22:43:09.454010 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.453933 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.456294 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.456259 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 22:43:09.456472 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.456320 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 22:43:09.456472 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.456457 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-spbmt\"" Jul 10 22:43:09.498218 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.498177 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:09.615241 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.615173 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a6372247-d610-4b24-83e0-b220e42ca4ab-crio-socket\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.615241 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.615231 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6372247-d610-4b24-83e0-b220e42ca4ab-data-volume\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.615532 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.615323 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c66zt\" (UniqueName: \"kubernetes.io/projected/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-api-access-c66zt\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.615532 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.615404 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.615532 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.615523 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716146 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716032 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716154 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a6372247-d610-4b24-83e0-b220e42ca4ab-crio-socket\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716191 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6372247-d610-4b24-83e0-b220e42ca4ab-data-volume\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716374 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:09.716245 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 22:43:09.716374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716271 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c66zt\" (UniqueName: \"kubernetes.io/projected/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-api-access-c66zt\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716282 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a6372247-d610-4b24-83e0-b220e42ca4ab-crio-socket\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716374 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:09.716334 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:10.216309388 +0000 UTC m=+48.853735061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : secret "insights-runtime-extractor-tls" not found Jul 10 22:43:09.716705 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716395 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.716705 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:09.716489 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 22:43:09.716705 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:09.716531 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:10.216518653 +0000 UTC m=+48.853944308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : configmap "kube-rbac-proxy" not found Jul 10 22:43:09.716705 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.716620 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6372247-d610-4b24-83e0-b220e42ca4ab-data-volume\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.728922 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.728885 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c66zt\" (UniqueName: \"kubernetes.io/projected/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-api-access-c66zt\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:09.885785 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.885742 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-4lvqn"] Jul 10 22:43:09.907597 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.907561 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-4lvqn"] Jul 10 22:43:09.907802 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.907709 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:09.910182 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.910154 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 10 22:43:09.910417 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.910391 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5qkwf\"" Jul 10 22:43:09.910481 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:09.910436 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 10 22:43:10.009729 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.009695 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:10.019946 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.019912 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b235a06e-ca2f-406a-ac42-1810e20b6f7c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:10.020072 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.019994 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:10.108636 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.108596 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:10.120954 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.120897 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b235a06e-ca2f-406a-ac42-1810e20b6f7c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:10.121158 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.121012 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:10.121291 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.121269 2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 22:43:10.121440 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.121346 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert podName:b235a06e-ca2f-406a-ac42-1810e20b6f7c nodeName:}" failed. No retries permitted until 2026-07-10 22:43:10.621325034 +0000 UTC m=+49.258750701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-4lvqn" (UID: "b235a06e-ca2f-406a-ac42-1810e20b6f7c") : secret "networking-console-plugin-cert" not found Jul 10 22:43:10.121861 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.121837 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b235a06e-ca2f-406a-ac42-1810e20b6f7c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:10.209253 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.209197 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:10.222012 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.221965 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:10.222205 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.222072 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:10.222205 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.222105 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 22:43:10.222205 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.222196 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:11.222170428 +0000 UTC m=+49.859596091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : configmap "kube-rbac-proxy" not found Jul 10 22:43:10.222399 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.222245 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 22:43:10.222399 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.222308 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:11.222290782 +0000 UTC m=+49.859716435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : secret "insights-runtime-extractor-tls" not found Jul 10 22:43:10.308453 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.308355 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:10.393491 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.393459 2551 scope.go:117] "RemoveContainer" containerID="7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb" Jul 10 22:43:10.393874 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.393732 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kk4fj_openshift-console-operator(09c729d5-a628-44ee-aaff-979c8a614328)\"" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podUID="09c729d5-a628-44ee-aaff-979c8a614328" Jul 10 22:43:10.531093 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.531045 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-sqvpx"] Jul 10 22:43:10.553503 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.553463 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-sqvpx"] Jul 10 22:43:10.553701 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.553603 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.556167 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.556138 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 10 22:43:10.556350 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.556174 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 10 22:43:10.556350 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.556269 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-l9bgl\"" Jul 10 22:43:10.556448 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.556138 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 10 22:43:10.556448 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.556138 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 10 22:43:10.565078 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.565036 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:10.624372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.624332 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:10.627555 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.627525 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-signing-cabundle\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.627688 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.627657 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4mcq\" (UniqueName: \"kubernetes.io/projected/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-kube-api-access-m4mcq\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.627762 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.627730 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-signing-key\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.627815 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.627789 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:10.627930 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.627912 2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 22:43:10.628010 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:10.627999 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert podName:b235a06e-ca2f-406a-ac42-1810e20b6f7c nodeName:}" failed. No retries permitted until 2026-07-10 22:43:11.627978539 +0000 UTC m=+50.265404196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-4lvqn" (UID: "b235a06e-ca2f-406a-ac42-1810e20b6f7c") : secret "networking-console-plugin-cert" not found Jul 10 22:43:10.728329 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.728284 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-signing-key\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.728547 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.728439 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-signing-cabundle\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.728547 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.728519 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mcq\" (UniqueName: \"kubernetes.io/projected/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-kube-api-access-m4mcq\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.729169 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.729147 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-signing-cabundle\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.730880 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.730851 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-signing-key\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.737361 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.737330 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4mcq\" (UniqueName: \"kubernetes.io/projected/8e9e7ca2-766c-43c0-99f7-e99907ad3d0b-kube-api-access-m4mcq\") pod \"service-ca-79f68b9f4f-sqvpx\" (UID: \"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b\") " pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:10.866361 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:10.866253 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" Jul 10 22:43:11.166202 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.166162 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-sqvpx"] Jul 10 22:43:11.170088 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:11.170057 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9e7ca2_766c_43c0_99f7_e99907ad3d0b.slice/crio-731720b5f730d3c88c12a8f198b9b149b24901bec392a8996bf1e302374a8163 WatchSource:0}: Error finding container 731720b5f730d3c88c12a8f198b9b149b24901bec392a8996bf1e302374a8163: Status 404 returned error can't find the container with id 731720b5f730d3c88c12a8f198b9b149b24901bec392a8996bf1e302374a8163 Jul 10 22:43:11.231460 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.231313 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:11.231460 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.231418 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:11.231597 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:11.231475 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 22:43:11.231597 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:11.231551 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.231528122 +0000 UTC m=+51.868953781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : configmap "kube-rbac-proxy" not found Jul 10 22:43:11.231717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:11.231601 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 22:43:11.231717 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:11.231644 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.231632651 +0000 UTC m=+51.869058304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : secret "insights-runtime-extractor-tls" not found Jul 10 22:43:11.300859 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.300820 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:11.398954 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.398914 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" event={"ID":"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b","Type":"ContainerStarted","Data":"d58001e5a71d84a4aa524c49b5921b769892f701c5e27b9091cb62cfb99f66f4"} Jul 10 22:43:11.399452 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.398982 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" event={"ID":"8e9e7ca2-766c-43c0-99f7-e99907ad3d0b","Type":"ContainerStarted","Data":"731720b5f730d3c88c12a8f198b9b149b24901bec392a8996bf1e302374a8163"} Jul 10 22:43:11.400906 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.400876 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" event={"ID":"734bd34c-bb9b-4254-9e81-71ccce05eec8","Type":"ContainerStarted","Data":"5d01a32bcbbcbbc206297b8fff260dba7660a47725f9030bc072fe0aaca9b2c8"} Jul 10 22:43:11.401075 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.400939 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" event={"ID":"734bd34c-bb9b-4254-9e81-71ccce05eec8","Type":"ContainerStarted","Data":"272011965ae3a9d9d15163ba66c877a1827698b5189a9a027193d161a71335ee"} Jul 10 22:43:11.418834 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.418709 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-sqvpx" podStartSLOduration=1.418687189 podStartE2EDuration="1.418687189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:11.418365002 +0000 UTC m=+50.055790681" watchObservedRunningTime="2026-07-10 22:43:11.418687189 +0000 UTC m=+50.056112867" Jul 10 22:43:11.436953 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.436896 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5ng5s" podStartSLOduration=1.5437449829999998 podStartE2EDuration="3.436872472s" totalImagesPullingTime="1.893127489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:43:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:11.436455017 +0000 UTC m=+50.073880693" watchObservedRunningTime="2026-07-10 22:43:11.436872472 +0000 UTC m=+50.074298147" Jul 10 22:43:11.606530 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.606497 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wm5gq_098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7/dns-node-resolver/0.log" Jul 10 22:43:11.634871 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:11.634833 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:11.635044 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:11.635022 2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 22:43:11.635127 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:11.635113 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert podName:b235a06e-ca2f-406a-ac42-1810e20b6f7c nodeName:}" failed. No retries permitted until 2026-07-10 22:43:13.635089708 +0000 UTC m=+52.272515367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-4lvqn" (UID: "b235a06e-ca2f-406a-ac42-1810e20b6f7c") : secret "networking-console-plugin-cert" not found Jul 10 22:43:12.545288 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:12.545249 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:12.603195 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:12.603165 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rx6mq_0f40b374-3519-464b-b29e-c3c2303f7fa8/node-ca/0.log" Jul 10 22:43:12.617635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:12.617596 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:12.780084 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:12.780048 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.010676 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.010642 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dj8lx_b76215f8-9338-4d83-a9e2-62779b4f100f/serve-healthcheck-canary/0.log" Jul 10 22:43:13.043706 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.043672 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.148650 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.148612 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.203930 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.203890 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5ng5s_734bd34c-bb9b-4254-9e81-71ccce05eec8/migrator/0.log" Jul 10 22:43:13.222050 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.222012 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.250985 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.250941 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:13.251204 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.251037 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:13.251204 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:13.251092 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 22:43:13.251361 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:13.251221 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 22:43:13.251633 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:13.251610 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:17.251162014 +0000 UTC m=+55.888587677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : configmap "kube-rbac-proxy" not found Jul 10 22:43:13.251779 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:13.251652 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:17.251638914 +0000 UTC m=+55.889064575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : secret "insights-runtime-extractor-tls" not found Jul 10 22:43:13.291781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.291676 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.392718 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.392671 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.406569 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.406507 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5ng5s_734bd34c-bb9b-4254-9e81-71ccce05eec8/graceful-termination/0.log" Jul 10 22:43:13.595094 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.594986 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:13.605476 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.605439 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-lg7dx_2b6daf41-2288-4cbf-ac15-2e333b6870ce/kube-storage-version-migrator-operator/0.log" Jul 10 22:43:13.656205 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.656159 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:13.656408 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:13.656351 2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 22:43:13.656460 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:13.656446 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert podName:b235a06e-ca2f-406a-ac42-1810e20b6f7c nodeName:}" failed. No retries permitted until 2026-07-10 22:43:17.656420156 +0000 UTC m=+56.293845821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-4lvqn" (UID: "b235a06e-ca2f-406a-ac42-1810e20b6f7c") : secret "networking-console-plugin-cert" not found Jul 10 22:43:13.854156 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:13.854056 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:14.018886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.018844 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:14.059878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.059838 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:14.059878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.059879 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.059939 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.059970 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.059998 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.060010 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060020 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787445c87-fv27n: secret "image-registry-tls" not found Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060085 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls podName:4f5403e3-e48e-4ec1-9d37-9553b9689ca5 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:30.060063317 +0000 UTC m=+68.697488970 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls") pod "image-registry-787445c87-fv27n" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5") : secret "image-registry-tls" not found Jul 10 22:43:14.060107 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060093 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060129 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060105 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:30.060095373 +0000 UTC m=+68.697521059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : configmap references non-existent config key: service-ca.crt Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060170 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls podName:354a05bd-82c0-4502-8e66-468ad5da070a nodeName:}" failed. No retries permitted until 2026-07-10 22:43:30.060157892 +0000 UTC m=+68.697583555 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls") pod "dns-default-4n7g4" (UID: "354a05bd-82c0-4502-8e66-468ad5da070a") : secret "dns-default-metrics-tls" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060200 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs podName:9e113a39-f272-431f-a6e5-818d5a373fc8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:30.060190413 +0000 UTC m=+68.697616069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs") pod "router-default-7b57bc954d-h2spw" (UID: "9e113a39-f272-431f-a6e5-818d5a373fc8") : secret "router-metrics-certs-default" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.060261 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.060297 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060392 2551 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060405 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060417 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b58f5dd-wvj84: secret "image-registry-tls" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060445 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls podName:90b8a8ae-1d63-4e73-8d39-2821e3ec325b nodeName:}" failed. No retries permitted until 2026-07-10 22:43:30.0604285 +0000 UTC m=+68.697854160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p5drw" (UID: "90b8a8ae-1d63-4e73-8d39-2821e3ec325b") : secret "cluster-monitoring-operator-tls" not found Jul 10 22:43:14.060490 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:14.060464 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls podName:a54c39d3-2d7e-4795-8043-c13457d5f0e8 nodeName:}" failed. No retries permitted until 2026-07-10 22:43:30.060453765 +0000 UTC m=+68.697879424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls") pod "image-registry-6b6b58f5dd-wvj84" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8") : secret "image-registry-tls" not found Jul 10 22:43:14.062528 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.062507 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/20ba6959-4ccb-41fc-be50-60195e103f97-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-74wz4\" (UID: \"20ba6959-4ccb-41fc-be50-60195e103f97\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:43:14.078755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.078708 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:14.113102 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.113005 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" Jul 10 22:43:14.270807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.270567 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4"] Jul 10 22:43:14.414293 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.414179 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" event={"ID":"20ba6959-4ccb-41fc-be50-60195e103f97","Type":"ContainerStarted","Data":"b7bca6db5778ed0b8b2cdc3badb3e280fac8c345edab359758d980704c755d40"} Jul 10 22:43:14.676481 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.676368 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:14.868765 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.868670 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:14.969993 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:14.969901 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:15.086774 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:15.086730 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:15.280403 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:15.280366 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:15.488943 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:15.488901 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:15.681368 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:15.681266 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:16.281191 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:16.281153 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:16.481401 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:16.481331 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:16.582245 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:16.582120 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:16.683363 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:16.683315 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:17.081233 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.081177 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:17.183549 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.183445 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:17.284423 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.284381 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:17.297557 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.297512 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:17.297776 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.297626 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:17.297830 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:17.297808 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 22:43:17.297905 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:17.297894 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls podName:a6372247-d610-4b24-83e0-b220e42ca4ab nodeName:}" failed. No retries permitted until 2026-07-10 22:43:25.297869159 +0000 UTC m=+63.935294820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nzs4r" (UID: "a6372247-d610-4b24-83e0-b220e42ca4ab") : secret "insights-runtime-extractor-tls" not found Jul 10 22:43:17.298079 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.298060 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a6372247-d610-4b24-83e0-b220e42ca4ab-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:17.429225 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.429182 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" event={"ID":"20ba6959-4ccb-41fc-be50-60195e103f97","Type":"ContainerStarted","Data":"1c1b3dcedac43de2dd895333c3160826e41e40e2b5977ad1d82acb1ccf80026c"} Jul 10 22:43:17.429430 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.429269 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" event={"ID":"20ba6959-4ccb-41fc-be50-60195e103f97","Type":"ContainerStarted","Data":"5f18b0a4d993aa0aeda78e6c8a99546b4d7fa9d45b2fba9d9e8ad44f0e1e7704"} Jul 10 22:43:17.447483 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.447343 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-74wz4" podStartSLOduration=36.140761903 podStartE2EDuration="38.447320444s" totalImagesPullingTime="2.306558541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:17.446701514 +0000 UTC m=+56.084127188" watchObservedRunningTime="2026-07-10 22:43:17.447320444 +0000 UTC m=+56.084746120" Jul 10 22:43:17.671641 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.671602 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:17.702033 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.701919 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:17.702565 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:17.702098 2551 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 22:43:17.702565 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:17.702192 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert podName:b235a06e-ca2f-406a-ac42-1810e20b6f7c nodeName:}" failed. No retries permitted until 2026-07-10 22:43:25.702167192 +0000 UTC m=+64.339592868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-4lvqn" (UID: "b235a06e-ca2f-406a-ac42-1810e20b6f7c") : secret "networking-console-plugin-cert" not found Jul 10 22:43:17.772511 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:17.772469 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:18.080600 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:18.080543 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:18.475666 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:18.475561 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:19.075054 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:19.075011 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:19.175470 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:19.175432 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:19.480231 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:19.480109 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:19.682925 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:19.682883 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:20.077651 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:20.077609 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:20.681767 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:20.681719 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:21.079450 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:21.079410 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:21.485278 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:21.485158 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:21.677261 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:21.677200 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:22.285102 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:22.285058 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:22.698658 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:22.698546 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:23.085729 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:23.083391 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:23.242423 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:23.242394 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5lnkm" Jul 10 22:43:23.280263 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:23.280222 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:23.481729 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:23.481623 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:24.035824 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.035783 2551 scope.go:117] "RemoveContainer" containerID="7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb" Jul 10 22:43:24.454265 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.454162 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:43:24.454577 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.454561 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/1.log" Jul 10 22:43:24.454634 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.454596 2551 generic.go:356] "Generic (PLEG): container finished" podID="09c729d5-a628-44ee-aaff-979c8a614328" containerID="b62992c8513abd5aeb45212dc7f44ddaadfa95a46cbee04e451901d5d007646f" exitCode=255 Jul 10 22:43:24.454743 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.454639 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kk4fj" event={"ID":"09c729d5-a628-44ee-aaff-979c8a614328","Type":"ContainerDied","Data":"b62992c8513abd5aeb45212dc7f44ddaadfa95a46cbee04e451901d5d007646f"} Jul 10 22:43:24.454743 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.454688 2551 scope.go:117] "RemoveContainer" containerID="7c897fbee903cef59cb11014210d505e9bdbe85680a14309f8637491e8b0d9cb" Jul 10 22:43:24.455075 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:24.455054 2551 scope.go:117] "RemoveContainer" containerID="b62992c8513abd5aeb45212dc7f44ddaadfa95a46cbee04e451901d5d007646f" Jul 10 22:43:24.455313 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:24.455288 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-kk4fj_openshift-console-operator(09c729d5-a628-44ee-aaff-979c8a614328)\"" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podUID="09c729d5-a628-44ee-aaff-979c8a614328" Jul 10 22:43:25.378653 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.378593 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:25.381087 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.381063 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a6372247-d610-4b24-83e0-b220e42ca4ab-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nzs4r\" (UID: \"a6372247-d610-4b24-83e0-b220e42ca4ab\") " pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:25.459692 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.459663 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:43:25.666848 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.666738 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-spbmt\"" Jul 10 22:43:25.675018 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.674987 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nzs4r" Jul 10 22:43:25.783503 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.783464 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:25.785985 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.785955 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b235a06e-ca2f-406a-ac42-1810e20b6f7c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-4lvqn\" (UID: \"b235a06e-ca2f-406a-ac42-1810e20b6f7c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:25.811838 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.811809 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nzs4r"] Jul 10 22:43:25.814482 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:25.814443 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6372247_d610_4b24_83e0_b220e42ca4ab.slice/crio-2f1601dc98bb6745bfd4aa686d9199cdae15ad87fe6c640ace7d4321c4892208 WatchSource:0}: Error finding container 2f1601dc98bb6745bfd4aa686d9199cdae15ad87fe6c640ace7d4321c4892208: Status 404 returned error can't find the container with id 2f1601dc98bb6745bfd4aa686d9199cdae15ad87fe6c640ace7d4321c4892208 Jul 10 22:43:25.821622 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.821596 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5qkwf\"" Jul 10 22:43:25.829694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.829666 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" Jul 10 22:43:25.969302 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:25.969197 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-4lvqn"] Jul 10 22:43:25.972416 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:25.972387 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb235a06e_ca2f_406a_ac42_1810e20b6f7c.slice/crio-ba25ce29c980e2e5dc39daeb34213d17fc87fed799244a897cedaa6489b51417 WatchSource:0}: Error finding container ba25ce29c980e2e5dc39daeb34213d17fc87fed799244a897cedaa6489b51417: Status 404 returned error can't find the container with id ba25ce29c980e2e5dc39daeb34213d17fc87fed799244a897cedaa6489b51417 Jul 10 22:43:26.466813 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.466768 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" event={"ID":"b235a06e-ca2f-406a-ac42-1810e20b6f7c","Type":"ContainerStarted","Data":"ba25ce29c980e2e5dc39daeb34213d17fc87fed799244a897cedaa6489b51417"} Jul 10 22:43:26.469029 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.468989 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzs4r" event={"ID":"a6372247-d610-4b24-83e0-b220e42ca4ab","Type":"ContainerStarted","Data":"2f1601dc98bb6745bfd4aa686d9199cdae15ad87fe6c640ace7d4321c4892208"} Jul 10 22:43:26.895399 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.895350 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:43:26.895616 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.895416 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:43:26.895616 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.895546 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:43:26.898823 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.898331 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 22:43:26.898823 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.898634 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 22:43:26.900602 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.900541 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jz5w\" (UniqueName: \"kubernetes.io/projected/736795d7-52c1-4a00-98c9-6b0caf6eb661-kube-api-access-4jz5w\") pod \"network-check-target-jxrxm\" (UID: \"736795d7-52c1-4a00-98c9-6b0caf6eb661\") " pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:43:26.909515 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.909475 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/931fa3c6-353f-43b5-9016-bfae335b8cc7-metrics-certs\") pod \"network-metrics-daemon-28nzx\" (UID: \"931fa3c6-353f-43b5-9016-bfae335b8cc7\") " pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:43:26.911170 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.911099 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60803b6d-b5c5-4c66-b602-f4e7c229ce13-original-pull-secret\") pod \"global-pull-secret-syncer-qgkfr\" (UID: \"60803b6d-b5c5-4c66-b602-f4e7c229ce13\") " pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:43:26.956694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.956652 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wwzq4\"" Jul 10 22:43:26.964955 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.964720 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:43:26.971617 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.971590 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vx2kj\"" Jul 10 22:43:26.976885 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.976826 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qgkfr" Jul 10 22:43:26.979654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:26.979625 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28nzx" Jul 10 22:43:27.405863 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.405830 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-28nzx"] Jul 10 22:43:27.408221 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:27.408166 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931fa3c6_353f_43b5_9016_bfae335b8cc7.slice/crio-c876a06a6957f35caa8804b1623ab4e93cff360a2d49214cd22e8c56a029ad24 WatchSource:0}: Error finding container c876a06a6957f35caa8804b1623ab4e93cff360a2d49214cd22e8c56a029ad24: Status 404 returned error can't find the container with id c876a06a6957f35caa8804b1623ab4e93cff360a2d49214cd22e8c56a029ad24 Jul 10 22:43:27.417242 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.417196 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qgkfr"] Jul 10 22:43:27.421250 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:27.421191 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60803b6d_b5c5_4c66_b602_f4e7c229ce13.slice/crio-1b295d1062e42add39c23c3aa859547f07b85fd06a3e090369e118503f2c90ae WatchSource:0}: Error finding container 1b295d1062e42add39c23c3aa859547f07b85fd06a3e090369e118503f2c90ae: Status 404 returned error can't find the container with id 1b295d1062e42add39c23c3aa859547f07b85fd06a3e090369e118503f2c90ae Jul 10 22:43:27.444606 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.444575 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jxrxm"] Jul 10 22:43:27.447543 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:27.447510 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736795d7_52c1_4a00_98c9_6b0caf6eb661.slice/crio-0c452633cb39e2e795e2771e5cc681a31bb6d7a9c277f4955fd768769d33050b WatchSource:0}: Error finding container 0c452633cb39e2e795e2771e5cc681a31bb6d7a9c277f4955fd768769d33050b: Status 404 returned error can't find the container with id 0c452633cb39e2e795e2771e5cc681a31bb6d7a9c277f4955fd768769d33050b Jul 10 22:43:27.474764 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.474726 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-28nzx" event={"ID":"931fa3c6-353f-43b5-9016-bfae335b8cc7","Type":"ContainerStarted","Data":"c876a06a6957f35caa8804b1623ab4e93cff360a2d49214cd22e8c56a029ad24"} Jul 10 22:43:27.476663 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.476616 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" event={"ID":"b235a06e-ca2f-406a-ac42-1810e20b6f7c","Type":"ContainerStarted","Data":"7009f8b2a07722c23470b2b587284ebfbed021d9dda7e66b587856fecbfb05e9"} Jul 10 22:43:27.478424 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.478393 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jxrxm" event={"ID":"736795d7-52c1-4a00-98c9-6b0caf6eb661","Type":"ContainerStarted","Data":"0c452633cb39e2e795e2771e5cc681a31bb6d7a9c277f4955fd768769d33050b"} Jul 10 22:43:27.479728 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.479693 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qgkfr" event={"ID":"60803b6d-b5c5-4c66-b602-f4e7c229ce13","Type":"ContainerStarted","Data":"1b295d1062e42add39c23c3aa859547f07b85fd06a3e090369e118503f2c90ae"} Jul 10 22:43:27.492752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:27.492699 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-4lvqn" podStartSLOduration=17.205614897 podStartE2EDuration="18.492679566s" totalImagesPullingTime="1.287064669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:27.490853337 +0000 UTC m=+66.128279013" watchObservedRunningTime="2026-07-10 22:43:27.492679566 +0000 UTC m=+66.130105241" Jul 10 22:43:28.486664 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:28.486619 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jxrxm" event={"ID":"736795d7-52c1-4a00-98c9-6b0caf6eb661","Type":"ContainerStarted","Data":"b30cfd08574b2115410cd8707e29ea597692257048c1b185a03d6aba346428a4"} Jul 10 22:43:28.503370 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:28.503308 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jxrxm" podStartSLOduration=66.503284469 podStartE2EDuration="1m6.503284469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:28.50228858 +0000 UTC m=+67.139714260" watchObservedRunningTime="2026-07-10 22:43:28.503284469 +0000 UTC m=+67.140710139" Jul 10 22:43:28.570114 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:28.570072 2551 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:43:28.570305 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:28.570139 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:43:28.570586 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:28.570569 2551 scope.go:117] "RemoveContainer" containerID="b62992c8513abd5aeb45212dc7f44ddaadfa95a46cbee04e451901d5d007646f" Jul 10 22:43:28.570813 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:28.570785 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-kk4fj_openshift-console-operator(09c729d5-a628-44ee-aaff-979c8a614328)\"" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podUID="09c729d5-a628-44ee-aaff-979c8a614328" Jul 10 22:43:29.481009 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.480944 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd"] Jul 10 22:43:29.492788 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.492754 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52"] Jul 10 22:43:29.493530 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.492877 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.495946 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.495918 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9cfzr\"" Jul 10 22:43:29.496467 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.496441 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 22:43:29.496608 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.496581 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 22:43:29.496711 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.496693 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 22:43:29.498108 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.497976 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 22:43:29.498361 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.498326 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:43:29.498479 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.498433 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.501604 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.501569 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 22:43:29.501775 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.501749 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 22:43:29.501960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.501939 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52"] Jul 10 22:43:29.503238 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.502654 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 22:43:29.503238 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.502678 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 22:43:29.505741 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.505676 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd"] Jul 10 22:43:29.515482 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.514563 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd"] Jul 10 22:43:29.518504 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.518471 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.522500 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.522305 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 22:43:29.536006 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.535971 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd"] Jul 10 22:43:29.596284 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.596247 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b6b58f5dd-wvj84"] Jul 10 22:43:29.596648 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:29.596606 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" podUID="a54c39d3-2d7e-4795-8043-c13457d5f0e8" Jul 10 22:43:29.623826 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.623781 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzthk\" (UniqueName: \"kubernetes.io/projected/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-kube-api-access-dzthk\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.624044 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.623892 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-hub\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.624044 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.623930 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx48r\" (UniqueName: \"kubernetes.io/projected/bdebe808-213b-478a-a1ec-385bfe6eca5c-kube-api-access-vx48r\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.624044 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.623979 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d73e874-3700-4b95-8d46-8f760bb7e2d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd\" (UID: \"4d73e874-3700-4b95-8d46-8f760bb7e2d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.624277 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624197 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdebe808-213b-478a-a1ec-385bfe6eca5c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.624277 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624268 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.624408 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624300 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-ca\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.624408 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624379 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.625585 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624407 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8km5q\" (UniqueName: \"kubernetes.io/projected/4d73e874-3700-4b95-8d46-8f760bb7e2d5-kube-api-access-8km5q\") pod \"managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd\" (UID: \"4d73e874-3700-4b95-8d46-8f760bb7e2d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.625585 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624440 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-klusterlet-config\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.625585 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.624462 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-tmp\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.631107 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.631071 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:29.722163 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.722120 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:29.725194 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725159 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d73e874-3700-4b95-8d46-8f760bb7e2d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd\" (UID: \"4d73e874-3700-4b95-8d46-8f760bb7e2d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.725419 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725252 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdebe808-213b-478a-a1ec-385bfe6eca5c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.725419 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725302 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.725419 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725336 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-ca\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.725419 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725409 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.725642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725430 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8km5q\" (UniqueName: \"kubernetes.io/projected/4d73e874-3700-4b95-8d46-8f760bb7e2d5-kube-api-access-8km5q\") pod \"managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd\" (UID: \"4d73e874-3700-4b95-8d46-8f760bb7e2d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.725642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725451 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-klusterlet-config\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.725642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725479 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-tmp\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.725642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725529 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzthk\" (UniqueName: \"kubernetes.io/projected/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-kube-api-access-dzthk\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.725642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725580 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-hub\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.725642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.725613 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx48r\" (UniqueName: \"kubernetes.io/projected/bdebe808-213b-478a-a1ec-385bfe6eca5c-kube-api-access-vx48r\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.727514 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.727476 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdebe808-213b-478a-a1ec-385bfe6eca5c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.727662 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.727547 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-tmp\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.728930 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.728876 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d73e874-3700-4b95-8d46-8f760bb7e2d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd\" (UID: \"4d73e874-3700-4b95-8d46-8f760bb7e2d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.730009 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.729976 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-klusterlet-config\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.730744 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.730715 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.730870 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.730762 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-ca\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.731589 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.731527 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-hub\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.732141 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.732114 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bdebe808-213b-478a-a1ec-385bfe6eca5c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.735739 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.735674 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8km5q\" (UniqueName: \"kubernetes.io/projected/4d73e874-3700-4b95-8d46-8f760bb7e2d5-kube-api-access-8km5q\") pod \"managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd\" (UID: \"4d73e874-3700-4b95-8d46-8f760bb7e2d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.736037 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.736003 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzthk\" (UniqueName: \"kubernetes.io/projected/e7fa2652-3d2b-4ae8-9da5-68eb7d52444d-kube-api-access-dzthk\") pod \"klusterlet-addon-workmgr-ff64b8b7f-kfrbd\" (UID: \"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.736538 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.736513 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx48r\" (UniqueName: \"kubernetes.io/projected/bdebe808-213b-478a-a1ec-385bfe6eca5c-kube-api-access-vx48r\") pod \"cluster-proxy-proxy-agent-98c6f84fd-lbj52\" (UID: \"bdebe808-213b-478a-a1ec-385bfe6eca5c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.816510 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.816464 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" Jul 10 22:43:29.826540 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.826433 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" Jul 10 22:43:29.834298 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.834253 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:29.853920 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.853881 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:29.930375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:29.930331 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:30.016187 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.016141 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:30.131894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.130805 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:30.131894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.130877 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:30.131894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.130955 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:43:30.131894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.130985 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:30.131894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.131045 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:30.131894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.131071 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:30.135814 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.135757 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e113a39-f272-431f-a6e5-818d5a373fc8-service-ca-bundle\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:30.144825 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.144787 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/354a05bd-82c0-4502-8e66-468ad5da070a-metrics-tls\") pod \"dns-default-4n7g4\" (UID: \"354a05bd-82c0-4502-8e66-468ad5da070a\") " pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:30.146725 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.146628 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"image-registry-6b6b58f5dd-wvj84\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:30.146725 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.146626 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e113a39-f272-431f-a6e5-818d5a373fc8-metrics-certs\") pod \"router-default-7b57bc954d-h2spw\" (UID: \"9e113a39-f272-431f-a6e5-818d5a373fc8\") " pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:30.149436 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.149345 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"image-registry-787445c87-fv27n\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:30.150572 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.150517 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/90b8a8ae-1d63-4e73-8d39-2821e3ec325b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p5drw\" (UID: \"90b8a8ae-1d63-4e73-8d39-2821e3ec325b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:43:30.162962 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.162859 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52"] Jul 10 22:43:30.178980 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.178942 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd"] Jul 10 22:43:30.182360 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.182324 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd"] Jul 10 22:43:30.277409 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.277368 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nwrhw\"" Jul 10 22:43:30.285630 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.285586 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" Jul 10 22:43:30.291661 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:30.291622 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdebe808_213b_478a_a1ec_385bfe6eca5c.slice/crio-734b7234c9e46a8e566139334380623b09443b372249178fe56c0bd4fbb278fc WatchSource:0}: Error finding container 734b7234c9e46a8e566139334380623b09443b372249178fe56c0bd4fbb278fc: Status 404 returned error can't find the container with id 734b7234c9e46a8e566139334380623b09443b372249178fe56c0bd4fbb278fc Jul 10 22:43:30.292610 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:30.292459 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d73e874_3700_4b95_8d46_8f760bb7e2d5.slice/crio-0c89f446aa444d753f501059425eabfd6385226a6454661f678b5115602f8e39 WatchSource:0}: Error finding container 0c89f446aa444d753f501059425eabfd6385226a6454661f678b5115602f8e39: Status 404 returned error can't find the container with id 0c89f446aa444d753f501059425eabfd6385226a6454661f678b5115602f8e39 Jul 10 22:43:30.293616 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:30.293451 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fa2652_3d2b_4ae8_9da5_68eb7d52444d.slice/crio-9684a9d864d8da4f4284f93ad5c4015561b84eaa113595c4b2e73485e9a9db8b WatchSource:0}: Error finding container 9684a9d864d8da4f4284f93ad5c4015561b84eaa113595c4b2e73485e9a9db8b: Status 404 returned error can't find the container with id 9684a9d864d8da4f4284f93ad5c4015561b84eaa113595c4b2e73485e9a9db8b Jul 10 22:43:30.348366 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.348332 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6t25n\"" Jul 10 22:43:30.356005 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.355961 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:30.356266 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.355962 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zw2wb\"" Jul 10 22:43:30.364425 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.364383 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:30.379742 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.379702 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j4q74\"" Jul 10 22:43:30.387817 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.387771 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:30.497137 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.497086 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" event={"ID":"4d73e874-3700-4b95-8d46-8f760bb7e2d5","Type":"ContainerStarted","Data":"0c89f446aa444d753f501059425eabfd6385226a6454661f678b5115602f8e39"} Jul 10 22:43:30.498582 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.498549 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" event={"ID":"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d","Type":"ContainerStarted","Data":"9684a9d864d8da4f4284f93ad5c4015561b84eaa113595c4b2e73485e9a9db8b"} Jul 10 22:43:30.499959 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.499922 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" event={"ID":"bdebe808-213b-478a-a1ec-385bfe6eca5c","Type":"ContainerStarted","Data":"734b7234c9e46a8e566139334380623b09443b372249178fe56c0bd4fbb278fc"} Jul 10 22:43:30.500110 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.499964 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:30.526764 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.526606 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:30.636010 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.635977 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a54c39d3-2d7e-4795-8043-c13457d5f0e8-ca-trust-extracted\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636010 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636021 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636285 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636057 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-image-registry-private-configuration\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636285 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636105 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-trusted-ca\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636285 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636130 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-certificates\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636285 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636170 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-installation-pull-secrets\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636285 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636243 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btlqg\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-kube-api-access-btlqg\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.636551 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.636338 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-bound-sa-token\") pod \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\" (UID: \"a54c39d3-2d7e-4795-8043-c13457d5f0e8\") " Jul 10 22:43:30.637309 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.637229 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 22:43:30.637309 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.637261 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 22:43:30.637627 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.637578 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a54c39d3-2d7e-4795-8043-c13457d5f0e8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 22:43:30.639816 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.639779 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:43:30.639963 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.639919 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 22:43:30.640463 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.640422 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:43:30.640922 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.640892 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-kube-api-access-btlqg" (OuterVolumeSpecName: "kube-api-access-btlqg") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "kube-api-access-btlqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:43:30.641678 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.641650 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a54c39d3-2d7e-4795-8043-c13457d5f0e8" (UID: "a54c39d3-2d7e-4795-8043-c13457d5f0e8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737076 2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-bound-sa-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737114 2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a54c39d3-2d7e-4795-8043-c13457d5f0e8-ca-trust-extracted\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737128 2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-tls\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737145 2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-image-registry-private-configuration\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737159 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-trusted-ca\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737171 2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a54c39d3-2d7e-4795-8043-c13457d5f0e8-registry-certificates\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737183 2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a54c39d3-2d7e-4795-8043-c13457d5f0e8-installation-pull-secrets\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.737227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.737196 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btlqg\" (UniqueName: \"kubernetes.io/projected/a54c39d3-2d7e-4795-8043-c13457d5f0e8-kube-api-access-btlqg\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:43:30.853454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:30.852944 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b57bc954d-h2spw"] Jul 10 22:43:31.093070 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.092971 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw"] Jul 10 22:43:31.100817 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.100539 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4n7g4"] Jul 10 22:43:31.103555 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.103181 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787445c87-fv27n"] Jul 10 22:43:31.119748 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:31.119706 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5403e3_e48e_4ec1_9d37_9553b9689ca5.slice/crio-5158bbdcd725187ec80027ce4f1f1ea54709ba867d470bf242441848489af368 WatchSource:0}: Error finding container 5158bbdcd725187ec80027ce4f1f1ea54709ba867d470bf242441848489af368: Status 404 returned error can't find the container with id 5158bbdcd725187ec80027ce4f1f1ea54709ba867d470bf242441848489af368 Jul 10 22:43:31.517789 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.516570 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-28nzx" event={"ID":"931fa3c6-353f-43b5-9016-bfae335b8cc7","Type":"ContainerStarted","Data":"55544deaf3fa07e0ffa0bb1e37d11cf270530f2e284d227737bc03777e89e0ca"} Jul 10 22:43:31.517789 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.516659 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-28nzx" event={"ID":"931fa3c6-353f-43b5-9016-bfae335b8cc7","Type":"ContainerStarted","Data":"822ff0e5d3edba01a780d5df0ef13e780bd886896c50d34f60816a3c2448441d"} Jul 10 22:43:31.528466 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.527673 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzs4r" event={"ID":"a6372247-d610-4b24-83e0-b220e42ca4ab","Type":"ContainerStarted","Data":"3d404ee596061df8b7e1ea13af7263c2cd3501d539cf3c64fe3dc878e8d1ed4f"} Jul 10 22:43:31.547916 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.547840 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-28nzx" podStartSLOduration=65.96153781699999 podStartE2EDuration="1m9.54781282s" totalImagesPullingTime="3.586275003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:31.542002177 +0000 UTC m=+70.179427853" watchObservedRunningTime="2026-07-10 22:43:31.54781282 +0000 UTC m=+70.185238496" Jul 10 22:43:31.553600 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.553256 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787445c87-fv27n" event={"ID":"4f5403e3-e48e-4ec1-9d37-9553b9689ca5","Type":"ContainerStarted","Data":"4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022"} Jul 10 22:43:31.553600 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.553332 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787445c87-fv27n" event={"ID":"4f5403e3-e48e-4ec1-9d37-9553b9689ca5","Type":"ContainerStarted","Data":"5158bbdcd725187ec80027ce4f1f1ea54709ba867d470bf242441848489af368"} Jul 10 22:43:31.554261 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.553881 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:43:31.558483 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.558443 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b57bc954d-h2spw" event={"ID":"9e113a39-f272-431f-a6e5-818d5a373fc8","Type":"ContainerStarted","Data":"1172ffd25bf0bf29483560a4efbadadce209ea09cf8bac0e593985f0b9d05a38"} Jul 10 22:43:31.558647 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.558516 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b57bc954d-h2spw" event={"ID":"9e113a39-f272-431f-a6e5-818d5a373fc8","Type":"ContainerStarted","Data":"4e67f3e34e759082fda12427caf9e4893ece0636e7aed38de772e704243edf86"} Jul 10 22:43:31.568925 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.568769 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" event={"ID":"90b8a8ae-1d63-4e73-8d39-2821e3ec325b","Type":"ContainerStarted","Data":"c2ad7004c4091d6e302b848b6b6c3d44671056f214e9c027d997276312dce0f7"} Jul 10 22:43:31.575403 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.575371 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b58f5dd-wvj84" Jul 10 22:43:31.578257 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.578157 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4n7g4" event={"ID":"354a05bd-82c0-4502-8e66-468ad5da070a","Type":"ContainerStarted","Data":"c64168df699869758f2eaaf51d0948fff97c3a462bf0c92a01b45bd31597077c"} Jul 10 22:43:31.581309 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.580952 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-787445c87-fv27n" podStartSLOduration=69.58092582 podStartE2EDuration="1m9.58092582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:31.578294944 +0000 UTC m=+70.215720628" watchObservedRunningTime="2026-07-10 22:43:31.58092582 +0000 UTC m=+70.218351495" Jul 10 22:43:31.610648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.609281 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b57bc954d-h2spw" podStartSLOduration=52.609255796 podStartE2EDuration="52.609255796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:31.608148241 +0000 UTC m=+70.245573910" watchObservedRunningTime="2026-07-10 22:43:31.609255796 +0000 UTC m=+70.246681464" Jul 10 22:43:31.641328 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.641255 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b6b58f5dd-wvj84"] Jul 10 22:43:31.659308 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.656035 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b6b58f5dd-wvj84"] Jul 10 22:43:31.659308 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.658003 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:31.721407 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:31.721329 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:32.052894 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:32.052616 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a54c39d3-2d7e-4795-8043-c13457d5f0e8" path="/var/lib/kubelet/pods/a54c39d3-2d7e-4795-8043-c13457d5f0e8/volumes" Jul 10 22:43:32.358096 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:32.357995 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:32.360199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:32.360162 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:32.592322 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:32.591623 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzs4r" event={"ID":"a6372247-d610-4b24-83e0-b220e42ca4ab","Type":"ContainerStarted","Data":"2bf15b0b6aeba78a2ea70a3112b64c2116a81380977508b4d2ef18da50b8d385"} Jul 10 22:43:32.594204 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:32.593501 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:32.596124 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:32.595868 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b57bc954d-h2spw" Jul 10 22:43:35.793123 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:35.792955 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:35.846263 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:35.846204 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:36.110032 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:36.109900 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:36.166534 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:36.166490 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:38.184563 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:38.184517 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:38.238924 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:38.238882 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:40.749594 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:40.749471 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:41.035803 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:41.035770 2551 scope.go:117] "RemoveContainer" containerID="b62992c8513abd5aeb45212dc7f44ddaadfa95a46cbee04e451901d5d007646f" Jul 10 22:43:41.036064 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:43:41.035979 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-kk4fj_openshift-console-operator(09c729d5-a628-44ee-aaff-979c8a614328)\"" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podUID="09c729d5-a628-44ee-aaff-979c8a614328" Jul 10 22:43:42.045390 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.045339 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:42.169489 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.169394 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:42.643039 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.642959 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" event={"ID":"90b8a8ae-1d63-4e73-8d39-2821e3ec325b","Type":"ContainerStarted","Data":"c6554d2c15f37986a379087ac13f60d033d11f17622de2f07d24f579e7394e84"} Jul 10 22:43:42.645519 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.645479 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4n7g4" event={"ID":"354a05bd-82c0-4502-8e66-468ad5da070a","Type":"ContainerStarted","Data":"9c4d23d57a3f28a5e4a90d1088048508958b8cffd65e1108f6e68567d46437c4"} Jul 10 22:43:42.645728 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.645564 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:42.645728 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.645595 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4n7g4" event={"ID":"354a05bd-82c0-4502-8e66-468ad5da070a","Type":"ContainerStarted","Data":"74a9cb5314199aa29d284fba485c113263319adb017f0ffea2cdb21c7719135b"} Jul 10 22:43:42.647134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.647105 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" event={"ID":"4d73e874-3700-4b95-8d46-8f760bb7e2d5","Type":"ContainerStarted","Data":"ca02ca2d28788bd3340e4904093bb99fbbc88f279f3d4b6db6b47cce9a73b474"} Jul 10 22:43:42.649919 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.649885 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nzs4r" event={"ID":"a6372247-d610-4b24-83e0-b220e42ca4ab","Type":"ContainerStarted","Data":"5a0ef552c812ca41bb53f9e8f68d36be286288b84b6264842dd04cd56db467c6"} Jul 10 22:43:42.651624 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.651595 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" event={"ID":"e7fa2652-3d2b-4ae8-9da5-68eb7d52444d","Type":"ContainerStarted","Data":"495b29345ac490aa89c36b498f0992b56bda5ccbeea771e4516b19394614cde9"} Jul 10 22:43:42.651790 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.651762 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:42.653737 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.653702 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qgkfr" event={"ID":"60803b6d-b5c5-4c66-b602-f4e7c229ce13","Type":"ContainerStarted","Data":"479efd5900f7d84773d6a85474d78ce9c7e9e4fe2f30370eea94d5b184d0119a"} Jul 10 22:43:42.654244 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.654000 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" Jul 10 22:43:42.656408 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.656379 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" event={"ID":"bdebe808-213b-478a-a1ec-385bfe6eca5c","Type":"ContainerStarted","Data":"0d6bdf410f525356d2a049ee93cd874dc05e35907ac497194ad337b4826fa2d7"} Jul 10 22:43:42.656553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.656417 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" event={"ID":"bdebe808-213b-478a-a1ec-385bfe6eca5c","Type":"ContainerStarted","Data":"11a1641259df438f6be4a30105d98a59db5fabc492d24dd60dccbed81584b395"} Jul 10 22:43:42.656553 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.656433 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" event={"ID":"bdebe808-213b-478a-a1ec-385bfe6eca5c","Type":"ContainerStarted","Data":"3cc18adb434bc0d4fbf1375d6adb03619b01e71182bbe0942bbf47f55e22747a"} Jul 10 22:43:42.664876 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.664809 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p5drw" podStartSLOduration=52.762290965 podStartE2EDuration="1m3.664786303s" totalImagesPullingTime="10.902495338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.663976257 +0000 UTC m=+81.301401933" watchObservedRunningTime="2026-07-10 22:43:42.664786303 +0000 UTC m=+81.302211956" Jul 10 22:43:42.684151 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.684081 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-98c6f84fd-lbj52" podStartSLOduration=1.948778992 podStartE2EDuration="13.684059391s" totalImagesPullingTime="11.735280399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.682486531 +0000 UTC m=+81.319912408" watchObservedRunningTime="2026-07-10 22:43:42.684059391 +0000 UTC m=+81.321485066" Jul 10 22:43:42.691073 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.691031 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:42.702055 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.701995 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5cb8b58b8d-8zzdd" podStartSLOduration=1.985307944 podStartE2EDuration="13.701973224s" totalImagesPullingTime="11.71666528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.700373915 +0000 UTC m=+81.337799604" watchObservedRunningTime="2026-07-10 22:43:42.701973224 +0000 UTC m=+81.339398902" Jul 10 22:43:42.727616 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.727549 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qgkfr" podStartSLOduration=66.140458419 podStartE2EDuration="1m20.727527143s" totalImagesPullingTime="14.587068724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.727166233 +0000 UTC m=+81.364591925" watchObservedRunningTime="2026-07-10 22:43:42.727527143 +0000 UTC m=+81.364952818" Jul 10 22:43:42.747846 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.747788 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4n7g4" podStartSLOduration=33.848478598 podStartE2EDuration="44.747767635s" totalImagesPullingTime="10.899289037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.74643822 +0000 UTC m=+81.383863896" watchObservedRunningTime="2026-07-10 22:43:42.747767635 +0000 UTC m=+81.385193311" Jul 10 22:43:42.772442 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.772321 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nzs4r" podStartSLOduration=17.626265406999998 podStartE2EDuration="33.772299962s" totalImagesPullingTime="16.146034555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:43:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.770382579 +0000 UTC m=+81.407808253" watchObservedRunningTime="2026-07-10 22:43:42.772299962 +0000 UTC m=+81.409725636" Jul 10 22:43:42.801865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.801799 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-ff64b8b7f-kfrbd" podStartSLOduration=3.374186702 podStartE2EDuration="13.801779487s" totalImagesPullingTime="10.427592785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:42.798622475 +0000 UTC m=+81.436048149" watchObservedRunningTime="2026-07-10 22:43:42.801779487 +0000 UTC m=+81.439205161" Jul 10 22:43:42.899788 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:42.899733 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:43.010273 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:43.010232 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:43.992533 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:43.992312 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:44.072757 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:44.072710 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:46.928446 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:46.928405 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:47.029228 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:47.029160 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:47.110975 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:47.110937 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:49.641352 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:49.641306 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:50.369523 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:50.369481 2551 patch_prober.go:28] interesting pod/image-registry-787445c87-fv27n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 22:43:50.369705 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:50.369570 2551 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787445c87-fv27n" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 22:43:51.057678 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.057643 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:51.269921 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.269881 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:51.295819 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.295783 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xc4rg"] Jul 10 22:43:51.302771 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.302738 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.305228 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.305181 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 22:43:51.305415 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.305240 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jwtnd\"" Jul 10 22:43:51.305532 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.305515 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 22:43:51.305611 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.305587 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 22:43:51.306258 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.306238 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 22:43:51.323537 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323492 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nd7x\" (UniqueName: \"kubernetes.io/projected/dc13cbb9-e635-4295-9d31-79caba9dc5b6-kube-api-access-7nd7x\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323548 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc13cbb9-e635-4295-9d31-79caba9dc5b6-metrics-client-ca\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323576 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-textfile\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323603 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323646 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-tls\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323668 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-wtmp\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323698 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-accelerators-collector-config\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.323755 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-sys\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.324010 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.323773 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-root\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.359568 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.359515 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:51.425071 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425029 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-root\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425435 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425088 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nd7x\" (UniqueName: \"kubernetes.io/projected/dc13cbb9-e635-4295-9d31-79caba9dc5b6-kube-api-access-7nd7x\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425435 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425116 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc13cbb9-e635-4295-9d31-79caba9dc5b6-metrics-client-ca\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425435 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425146 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-textfile\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425435 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425174 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425435 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425176 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-root\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425818 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425486 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-tls\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425818 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425594 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-textfile\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.425960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425932 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc13cbb9-e635-4295-9d31-79caba9dc5b6-metrics-client-ca\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.426034 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.425941 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-wtmp\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.426034 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.426014 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-accelerators-collector-config\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.426133 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.426047 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-sys\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.426133 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.426083 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-wtmp\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.426261 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.426155 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc13cbb9-e635-4295-9d31-79caba9dc5b6-sys\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.426671 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.426650 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-accelerators-collector-config\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.428550 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.428444 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-tls\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.428550 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.428497 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc13cbb9-e635-4295-9d31-79caba9dc5b6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.428550 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.428539 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:51.439081 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.439039 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nd7x\" (UniqueName: \"kubernetes.io/projected/dc13cbb9-e635-4295-9d31-79caba9dc5b6-kube-api-access-7nd7x\") pod \"node-exporter-xc4rg\" (UID: \"dc13cbb9-e635-4295-9d31-79caba9dc5b6\") " pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.572029 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.571917 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:51.617792 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.617746 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xc4rg" Jul 10 22:43:51.634171 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:43:51.634122 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc13cbb9_e635_4295_9d31_79caba9dc5b6.slice/crio-964027fd553eb018f84222a1fd8224d765641b4d475591b5a285ee9176d02253 WatchSource:0}: Error finding container 964027fd553eb018f84222a1fd8224d765641b4d475591b5a285ee9176d02253: Status 404 returned error can't find the container with id 964027fd553eb018f84222a1fd8224d765641b4d475591b5a285ee9176d02253 Jul 10 22:43:51.689361 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.689318 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:51.696969 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:51.696927 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xc4rg" event={"ID":"dc13cbb9-e635-4295-9d31-79caba9dc5b6","Type":"ContainerStarted","Data":"964027fd553eb018f84222a1fd8224d765641b4d475591b5a285ee9176d02253"} Jul 10 22:43:52.085194 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.085155 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:52.184397 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.184356 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:52.284784 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.284738 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:52.379241 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.379129 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:52.599784 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.599741 2551 patch_prober.go:28] interesting pod/image-registry-787445c87-fv27n container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 22:43:52.599957 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.599810 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787445c87-fv27n" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 22:43:52.663694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.663652 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4n7g4" Jul 10 22:43:52.819834 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:52.819777 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:53.708502 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:53.708467 2551 generic.go:356] "Generic (PLEG): container finished" podID="dc13cbb9-e635-4295-9d31-79caba9dc5b6" containerID="3ce8a17b9c46b284a12203bdcfaeb1e7e86085b7af1ac55e021aebee42a58c03" exitCode=0 Jul 10 22:43:53.708919 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:53.708540 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xc4rg" event={"ID":"dc13cbb9-e635-4295-9d31-79caba9dc5b6","Type":"ContainerDied","Data":"3ce8a17b9c46b284a12203bdcfaeb1e7e86085b7af1ac55e021aebee42a58c03"} Jul 10 22:43:54.323645 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.323605 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:54.380387 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.380349 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:54.450346 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.450307 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:54.583644 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.583543 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:54.714736 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.714692 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xc4rg" event={"ID":"dc13cbb9-e635-4295-9d31-79caba9dc5b6","Type":"ContainerStarted","Data":"d659a18078e14c4e640386e9bb0a53e376889f50fb630c6686a83fecc66a1307"} Jul 10 22:43:54.715242 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.714762 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xc4rg" event={"ID":"dc13cbb9-e635-4295-9d31-79caba9dc5b6","Type":"ContainerStarted","Data":"00e7feeedf73ee923c4e2db990992db216cc63724b8d7333e8beb975ab0395af"} Jul 10 22:43:54.736085 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.736023 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xc4rg" podStartSLOduration=2.64525861 podStartE2EDuration="3.736001991s" totalImagesPullingTime="1.021418922s" totalInitContainerRuntime="69.324459ms" isStatefulPod=true podCreationTimestamp="2026-07-10 22:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:54.734173202 +0000 UTC m=+93.371598877" watchObservedRunningTime="2026-07-10 22:43:54.736001991 +0000 UTC m=+93.373427665" Jul 10 22:43:54.817474 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.817436 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:54.918234 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:54.918126 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:55.019315 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.019273 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:55.036665 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.036634 2551 scope.go:117] "RemoveContainer" containerID="b62992c8513abd5aeb45212dc7f44ddaadfa95a46cbee04e451901d5d007646f" Jul 10 22:43:55.620095 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.620048 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:55.687458 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.687423 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:55.724859 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.724822 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:43:55.725367 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.724958 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kk4fj" event={"ID":"09c729d5-a628-44ee-aaff-979c8a614328","Type":"ContainerStarted","Data":"25b735cc275b310470d4405c20d909c3c3abc05a248dd8197d1b206a3c97d726"} Jul 10 22:43:55.743292 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.743232 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-kk4fj" podStartSLOduration=68.999809205 podStartE2EDuration="1m16.743198996s" totalImagesPullingTime="7.743389791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:43:55.74164918 +0000 UTC m=+94.379074868" watchObservedRunningTime="2026-07-10 22:43:55.743198996 +0000 UTC m=+94.380624670" Jul 10 22:43:55.746084 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.746045 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:55.906180 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:55.906077 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.084260 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.084199 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.184814 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.184686 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.284470 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.284425 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.384337 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.384271 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.439859 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.439733 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.679457 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.679402 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:56.732642 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:56.732542 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:57.493745 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:57.493692 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:57.599594 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:57.599551 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:57.693922 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:57.693879 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:57.950498 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:57.950458 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:58.098982 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.098939 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-787445c87-fv27n"] Jul 10 22:43:58.103652 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.103608 2551 patch_prober.go:28] interesting pod/image-registry-787445c87-fv27n container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 22:43:58.103815 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.103688 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787445c87-fv27n" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 22:43:58.180495 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.180455 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:58.381851 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.381819 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:58.435623 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.435584 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:58.503948 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.503910 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:58.604560 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:58.604517 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:59.872202 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:59.872157 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:43:59.966205 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:43:59.966162 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:00.097088 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:00.097048 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:00.198072 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:00.197972 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:00.503165 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:00.503134 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jxrxm" Jul 10 22:44:00.954002 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:00.953907 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:01.015480 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:01.015443 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:01.130944 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:01.130896 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:01.183670 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:01.183629 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:01.798471 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:01.798429 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:01.900814 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:01.900777 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:02.129881 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:02.129781 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:02.233556 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:02.233514 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:05.725530 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:05.725496 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:44:05.729566 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:05.729538 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-kk4fj" Jul 10 22:44:08.104415 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:08.104382 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:44:09.161561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:09.161512 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:09.239661 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:09.239612 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:10.794768 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:10.794694 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:10.845301 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:10.845263 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:10.915848 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:10.915810 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:11.017333 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:11.017283 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:11.120343 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:11.120241 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:11.214286 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:11.214244 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:11.279955 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:11.279920 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:14.703900 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:14.703854 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:14.766430 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:14.766376 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:16.513618 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:16.513577 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:16.616286 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:16.616188 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:17.384576 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:17.384539 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:17.443017 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:17.442969 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:21.710483 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:21.710436 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:21.811153 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:21.811112 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:23.120653 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.120606 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787445c87-fv27n" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" containerID="cri-o://4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022" gracePeriod=30 Jul 10 22:44:23.374914 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.374880 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:44:23.517559 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517520 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517572 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-installation-pull-secrets\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517625 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kztg2\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-kube-api-access-kztg2\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517752 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517712 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-bound-sa-token\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517760 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-certificates\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517797 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-trusted-ca\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517856 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-ca-trust-extracted\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.517918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.517881 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-image-registry-private-configuration\") pod \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\" (UID: \"4f5403e3-e48e-4ec1-9d37-9553b9689ca5\") " Jul 10 22:44:23.518299 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.518205 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 22:44:23.518299 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.518287 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 22:44:23.520201 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.520161 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:44:23.520456 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.520430 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 22:44:23.520728 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.520700 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 22:44:23.520856 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.520743 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:44:23.520856 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.520831 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-kube-api-access-kztg2" (OuterVolumeSpecName: "kube-api-access-kztg2") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "kube-api-access-kztg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:44:23.526370 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.526328 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4f5403e3-e48e-4ec1-9d37-9553b9689ca5" (UID: "4f5403e3-e48e-4ec1-9d37-9553b9689ca5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 22:44:23.619357 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619314 2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-bound-sa-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619357 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619350 2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-certificates\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619357 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619361 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-trusted-ca\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619357 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619370 2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-ca-trust-extracted\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619620 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619380 2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-image-registry-private-configuration\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619620 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619390 2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-registry-tls\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619620 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619398 2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-installation-pull-secrets\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.619620 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.619409 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kztg2\" (UniqueName: \"kubernetes.io/projected/4f5403e3-e48e-4ec1-9d37-9553b9689ca5-kube-api-access-kztg2\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:44:23.841258 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.841128 2551 generic.go:356] "Generic (PLEG): container finished" podID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerID="4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022" exitCode=0 Jul 10 22:44:23.841258 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.841180 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787445c87-fv27n" event={"ID":"4f5403e3-e48e-4ec1-9d37-9553b9689ca5","Type":"ContainerDied","Data":"4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022"} Jul 10 22:44:23.841498 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.841280 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787445c87-fv27n" Jul 10 22:44:23.841498 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.841305 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787445c87-fv27n" event={"ID":"4f5403e3-e48e-4ec1-9d37-9553b9689ca5","Type":"ContainerDied","Data":"5158bbdcd725187ec80027ce4f1f1ea54709ba867d470bf242441848489af368"} Jul 10 22:44:23.841498 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.841335 2551 scope.go:117] "RemoveContainer" containerID="4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022" Jul 10 22:44:23.843134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.843111 2551 generic.go:356] "Generic (PLEG): container finished" podID="2b6daf41-2288-4cbf-ac15-2e333b6870ce" containerID="8fa702938ba87d3a7068b6cd91d61a276da676dbf20182a05c5b5f1175bdbbdf" exitCode=0 Jul 10 22:44:23.843282 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.843148 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" event={"ID":"2b6daf41-2288-4cbf-ac15-2e333b6870ce","Type":"ContainerDied","Data":"8fa702938ba87d3a7068b6cd91d61a276da676dbf20182a05c5b5f1175bdbbdf"} Jul 10 22:44:23.843526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.843512 2551 scope.go:117] "RemoveContainer" containerID="8fa702938ba87d3a7068b6cd91d61a276da676dbf20182a05c5b5f1175bdbbdf" Jul 10 22:44:23.856583 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.856553 2551 scope.go:117] "RemoveContainer" containerID="4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022" Jul 10 22:44:23.856937 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:44:23.856913 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022\": container with ID starting with 4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022 not found: ID does not exist" containerID="4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022" Jul 10 22:44:23.857004 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.856948 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022"} err="failed to get container status \"4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022\": rpc error: code = NotFound desc = could not find container \"4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022\": container with ID starting with 4317d7e9cca981af119248a1a9e21ddd3b9b91525455a6da6c7b6462d4ec7022 not found: ID does not exist" Jul 10 22:44:23.879302 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.879191 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-787445c87-fv27n"] Jul 10 22:44:23.884614 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:23.884580 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-787445c87-fv27n"] Jul 10 22:44:24.042802 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:24.042756 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" path="/var/lib/kubelet/pods/4f5403e3-e48e-4ec1-9d37-9553b9689ca5/volumes" Jul 10 22:44:24.321965 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:24.321920 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:24.849069 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:24.849028 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-lg7dx" event={"ID":"2b6daf41-2288-4cbf-ac15-2e333b6870ce","Type":"ContainerStarted","Data":"2a191d68919c0faedf23c37c24f09487d38386f05a7dcec4d40073c55324fa35"} Jul 10 22:44:25.742625 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:25.742590 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:25.817680 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:25.817640 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:29.876279 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:29.876237 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:29.951054 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:29.951013 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:37.901795 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:37.901698 2551 generic.go:356] "Generic (PLEG): container finished" podID="61e7fa0b-2a53-430b-ae38-a7637a40a384" containerID="a66972e8ac70bb3676597ae642157151653b587b45b694e7c07fd1ad4f59e49c" exitCode=0 Jul 10 22:44:37.901795 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:37.901770 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" event={"ID":"61e7fa0b-2a53-430b-ae38-a7637a40a384","Type":"ContainerDied","Data":"a66972e8ac70bb3676597ae642157151653b587b45b694e7c07fd1ad4f59e49c"} Jul 10 22:44:37.902273 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:37.902170 2551 scope.go:117] "RemoveContainer" containerID="a66972e8ac70bb3676597ae642157151653b587b45b694e7c07fd1ad4f59e49c" Jul 10 22:44:38.907454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:38.907416 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gprfb" event={"ID":"61e7fa0b-2a53-430b-ae38-a7637a40a384","Type":"ContainerStarted","Data":"e4fd782cc479feec9810f222f081b2f64e5be309392a60b50f14884f671a8345"} Jul 10 22:44:38.908998 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:38.908968 2551 generic.go:356] "Generic (PLEG): container finished" podID="db6471ad-9a09-4ecc-8af9-8a0659846986" containerID="87f1eeee6ed9c401cd1f7aa83171e3b832357b5e41a3aa5e58dc0985574ec1d9" exitCode=0 Jul 10 22:44:38.909132 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:38.909014 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" event={"ID":"db6471ad-9a09-4ecc-8af9-8a0659846986","Type":"ContainerDied","Data":"87f1eeee6ed9c401cd1f7aa83171e3b832357b5e41a3aa5e58dc0985574ec1d9"} Jul 10 22:44:38.909348 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:38.909335 2551 scope.go:117] "RemoveContainer" containerID="87f1eeee6ed9c401cd1f7aa83171e3b832357b5e41a3aa5e58dc0985574ec1d9" Jul 10 22:44:39.914891 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:39.914851 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-p7ldl" event={"ID":"db6471ad-9a09-4ecc-8af9-8a0659846986","Type":"ContainerStarted","Data":"86307f48d940ed6c97690e9b63bfa744e57a323352a72cc4d13fefef91ca8360"} Jul 10 22:44:40.603010 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:40.602978 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:40.711001 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:40.710958 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:44.864615 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:44.864571 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:44.928563 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:44.928527 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:50.820353 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:50.820302 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:51.018585 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:51.018549 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:51.120466 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:51.120335 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:51.555062 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:51.555013 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:44:51.618106 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:44:51.618020 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:45:01.398227 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:45:01.398185 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:45:01.501138 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:45:01.501022 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:45:01.599469 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:45:01.599426 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:45:55.850518 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:45:55.850466 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:45:55.950736 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:45:55.950700 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:05.356800 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:05.356757 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:05.413759 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:05.413711 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:07.751453 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:07.751418 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:07.809012 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:07.808980 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:09.729434 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:09.729400 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:09.781067 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:09.781024 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:14.869604 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:14.869564 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:14.967690 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:14.967653 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:15.022184 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:15.022156 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:16.652348 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:16.652305 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:16.743949 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:16.743913 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:16.798713 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:16.798678 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:38.805224 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:38.805175 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:38.902491 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:38.902453 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:46:40.024918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:46:40.024872 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:05.530366 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:05.530328 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:06.307513 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:06.307479 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:09.139098 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:09.139059 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:09.239173 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:09.239142 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:10.204192 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:10.204155 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:10.351974 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:10.351934 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:11.137995 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:11.137951 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:11.257581 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:11.257551 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:13.143014 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:13.142985 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:13.536037 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:13.536012 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:15.358991 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:15.358938 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:15.935668 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:15.935639 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:16.055256 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:16.055227 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:20.209609 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:20.209576 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:20.303439 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:20.303407 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:21.755197 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:21.755162 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:21.928415 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:21.928393 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:47:21.932918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:21.932898 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:47:21.940249 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:21.940195 2551 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 22:47:25.547867 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:25.547832 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:26.817270 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:26.817206 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:27.348470 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:27.348439 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:40.355602 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:40.355565 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:40.448047 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:40.448019 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:40.573220 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:40.573193 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:40.671195 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:40.671123 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:40.961015 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:40.960950 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.173635 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.173601 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.274076 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.274038 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.324356 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.324330 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5"] Jul 10 22:47:41.324630 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.324617 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" Jul 10 22:47:41.324673 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.324631 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" Jul 10 22:47:41.324707 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.324697 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5403e3-e48e-4ec1-9d37-9553b9689ca5" containerName="registry" Jul 10 22:47:41.327572 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.327557 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.330122 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.330104 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 22:47:41.330724 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.330707 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 22:47:41.331306 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.331283 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l7nx4\"" Jul 10 22:47:41.331865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.331847 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 22:47:41.332547 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.332530 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 22:47:41.354759 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.354733 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5"] Jul 10 22:47:41.382071 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.382045 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59109aa9-8f44-4050-97e0-8828281f6916-webhook-cert\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.382351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.382092 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59109aa9-8f44-4050-97e0-8828281f6916-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.382351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.382119 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59109aa9-8f44-4050-97e0-8828281f6916-manifests\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.382351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.382154 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59109aa9-8f44-4050-97e0-8828281f6916-tmp\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.382351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.382198 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6462x\" (UniqueName: \"kubernetes.io/projected/59109aa9-8f44-4050-97e0-8828281f6916-kube-api-access-6462x\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.442930 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.442905 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.482823 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.482798 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59109aa9-8f44-4050-97e0-8828281f6916-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.482910 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.482831 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59109aa9-8f44-4050-97e0-8828281f6916-manifests\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.482910 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.482851 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59109aa9-8f44-4050-97e0-8828281f6916-tmp\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.482910 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.482874 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6462x\" (UniqueName: \"kubernetes.io/projected/59109aa9-8f44-4050-97e0-8828281f6916-kube-api-access-6462x\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.482910 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.482900 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59109aa9-8f44-4050-97e0-8828281f6916-webhook-cert\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.483296 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.483277 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59109aa9-8f44-4050-97e0-8828281f6916-manifests\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.485149 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.485128 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59109aa9-8f44-4050-97e0-8828281f6916-tmp\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.485287 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.485271 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59109aa9-8f44-4050-97e0-8828281f6916-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.485341 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.485278 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59109aa9-8f44-4050-97e0-8828281f6916-webhook-cert\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.493814 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.493792 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6462x\" (UniqueName: \"kubernetes.io/projected/59109aa9-8f44-4050-97e0-8828281f6916-kube-api-access-6462x\") pod \"opendatahub-operator-controller-manager-845b6c684-vgpj5\" (UID: \"59109aa9-8f44-4050-97e0-8828281f6916\") " pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.544521 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.544463 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.637674 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.637652 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:41.644422 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.644401 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.747059 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.747023 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.775852 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.775831 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5"] Jul 10 22:47:41.780887 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:47:41.780840 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59109aa9_8f44_4050_97e0_8828281f6916.slice/crio-6b33a11b38fdfe0e9caf69375d79123ee764729e972dbd780a55d3923e77d72e WatchSource:0}: Error finding container 6b33a11b38fdfe0e9caf69375d79123ee764729e972dbd780a55d3923e77d72e: Status 404 returned error can't find the container with id 6b33a11b38fdfe0e9caf69375d79123ee764729e972dbd780a55d3923e77d72e Jul 10 22:47:41.781672 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.781657 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 22:47:41.871731 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.871672 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:41.987841 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:41.987819 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:42.629047 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:42.629001 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" event={"ID":"59109aa9-8f44-4050-97e0-8828281f6916","Type":"ContainerStarted","Data":"6b33a11b38fdfe0e9caf69375d79123ee764729e972dbd780a55d3923e77d72e"} Jul 10 22:47:45.644251 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:45.644200 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" event={"ID":"59109aa9-8f44-4050-97e0-8828281f6916","Type":"ContainerStarted","Data":"00f7bc856b8aa426c078901b2762e3b8f36aab6b03d098c7700a0e16c8496dc4"} Jul 10 22:47:45.655330 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:45.655304 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:45.800886 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:45.800856 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:46.649629 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:46.649595 2551 generic.go:356] "Generic (PLEG): container finished" podID="59109aa9-8f44-4050-97e0-8828281f6916" containerID="00f7bc856b8aa426c078901b2762e3b8f36aab6b03d098c7700a0e16c8496dc4" exitCode=0 Jul 10 22:47:46.649990 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:46.649679 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" event={"ID":"59109aa9-8f44-4050-97e0-8828281f6916","Type":"ContainerDied","Data":"00f7bc856b8aa426c078901b2762e3b8f36aab6b03d098c7700a0e16c8496dc4"} Jul 10 22:47:47.655452 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:47.655421 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" event={"ID":"59109aa9-8f44-4050-97e0-8828281f6916","Type":"ContainerStarted","Data":"dca60139c3bc2e3536203a2beb89cbef58e0f93db6c47f2a2942ccf31bb1d8df"} Jul 10 22:47:47.655805 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:47.655519 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:47:47.683479 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:47.683438 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" podStartSLOduration=2.337338914 podStartE2EDuration="6.683424372s" totalImagesPullingTime="3.263268128s" totalInitContainerRuntime="1.08281733s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:47:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:47:47.681553033 +0000 UTC m=+326.318978707" watchObservedRunningTime="2026-07-10 22:47:47.683424372 +0000 UTC m=+326.320850049" Jul 10 22:47:53.021218 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:53.021185 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:53.081166 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:53.081139 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:47:58.661399 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:47:58.661370 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-845b6c684-vgpj5" Jul 10 22:48:00.241168 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:00.241135 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:00.410970 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:00.410933 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:01.137537 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.137508 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hb647"] Jul 10 22:48:01.146089 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.146071 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.148667 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.148645 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 22:48:01.148667 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.148664 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-c9skj\"" Jul 10 22:48:01.153130 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.153057 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hb647"] Jul 10 22:48:01.221138 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.221107 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p86w\" (UniqueName: \"kubernetes.io/projected/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-kube-api-access-8p86w\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.221293 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.221155 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-cert\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.245594 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.245567 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:01.321874 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.321843 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p86w\" (UniqueName: \"kubernetes.io/projected/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-kube-api-access-8p86w\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.322022 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.321892 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-cert\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.322022 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:48:01.321987 2551 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 22:48:01.322109 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:48:01.322046 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-cert podName:3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38 nodeName:}" failed. No retries permitted until 2026-07-10 22:48:01.822029466 +0000 UTC m=+340.459455118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-cert") pod "odh-model-controller-7998c67dc4-hb647" (UID: "3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38") : secret "odh-model-controller-webhook-cert" not found Jul 10 22:48:01.333767 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.333737 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p86w\" (UniqueName: \"kubernetes.io/projected/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-kube-api-access-8p86w\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.344952 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.344924 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:01.449019 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.448936 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:01.754079 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.754044 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:01.825370 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.825334 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-cert\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:01.827717 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:01.827690 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38-cert\") pod \"odh-model-controller-7998c67dc4-hb647\" (UID: \"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38\") " pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:02.056531 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.056436 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:02.056531 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.056459 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:02.100871 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.100842 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-2hp5d"] Jul 10 22:48:02.104009 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.103805 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.106544 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.106395 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4vf22\"" Jul 10 22:48:02.106649 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.106539 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 22:48:02.118687 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.118662 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-2hp5d"] Jul 10 22:48:02.227901 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.227873 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzrn9\" (UniqueName: \"kubernetes.io/projected/5544f4b6-217a-4264-9ce4-98a39eb48b3f-kube-api-access-xzrn9\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.228045 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.227911 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5544f4b6-217a-4264-9ce4-98a39eb48b3f-tls-certs\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.328317 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.328243 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzrn9\" (UniqueName: \"kubernetes.io/projected/5544f4b6-217a-4264-9ce4-98a39eb48b3f-kube-api-access-xzrn9\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.328317 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.328280 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5544f4b6-217a-4264-9ce4-98a39eb48b3f-tls-certs\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.328716 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:48:02.328380 2551 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 22:48:02.328716 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:48:02.328439 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5544f4b6-217a-4264-9ce4-98a39eb48b3f-tls-certs podName:5544f4b6-217a-4264-9ce4-98a39eb48b3f nodeName:}" failed. No retries permitted until 2026-07-10 22:48:02.828417771 +0000 UTC m=+341.465843429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5544f4b6-217a-4264-9ce4-98a39eb48b3f-tls-certs") pod "model-serving-api-54d89bd5fc-2hp5d" (UID: "5544f4b6-217a-4264-9ce4-98a39eb48b3f") : secret "model-serving-api-tls" not found Jul 10 22:48:02.341224 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.341188 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzrn9\" (UniqueName: \"kubernetes.io/projected/5544f4b6-217a-4264-9ce4-98a39eb48b3f-kube-api-access-xzrn9\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.411527 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.411501 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hb647"] Jul 10 22:48:02.413956 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:48:02.413928 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb8165b_ffd5_4b5c_9ee8_01e75ac11b38.slice/crio-7bbe85bafd6d1c1c88627d2592b3fc5d45c353b89319565f6baac961d03c7d23 WatchSource:0}: Error finding container 7bbe85bafd6d1c1c88627d2592b3fc5d45c353b89319565f6baac961d03c7d23: Status 404 returned error can't find the container with id 7bbe85bafd6d1c1c88627d2592b3fc5d45c353b89319565f6baac961d03c7d23 Jul 10 22:48:02.417549 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.417523 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:02.714109 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.714043 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" event={"ID":"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38","Type":"ContainerStarted","Data":"7bbe85bafd6d1c1c88627d2592b3fc5d45c353b89319565f6baac961d03c7d23"} Jul 10 22:48:02.832624 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.832602 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5544f4b6-217a-4264-9ce4-98a39eb48b3f-tls-certs\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:02.834844 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:02.834827 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5544f4b6-217a-4264-9ce4-98a39eb48b3f-tls-certs\") pod \"model-serving-api-54d89bd5fc-2hp5d\" (UID: \"5544f4b6-217a-4264-9ce4-98a39eb48b3f\") " pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:03.021649 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:03.021616 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:03.191322 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:03.191296 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-2hp5d"] Jul 10 22:48:03.193653 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:48:03.193623 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5544f4b6_217a_4264_9ce4_98a39eb48b3f.slice/crio-cfe86fb998d2e10a43d47aa14fbeb0f2865e3a36bc8e873d357b65610f42c949 WatchSource:0}: Error finding container cfe86fb998d2e10a43d47aa14fbeb0f2865e3a36bc8e873d357b65610f42c949: Status 404 returned error can't find the container with id cfe86fb998d2e10a43d47aa14fbeb0f2865e3a36bc8e873d357b65610f42c949 Jul 10 22:48:03.721362 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:03.721322 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" event={"ID":"5544f4b6-217a-4264-9ce4-98a39eb48b3f","Type":"ContainerStarted","Data":"cfe86fb998d2e10a43d47aa14fbeb0f2865e3a36bc8e873d357b65610f42c949"} Jul 10 22:48:05.734856 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:05.734773 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" event={"ID":"3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38","Type":"ContainerStarted","Data":"c0ad65c63776424746907da37896195e60708fc8b03bcabe2a1c6fd475146904"} Jul 10 22:48:05.735312 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:05.734870 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:05.736767 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:05.736741 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" event={"ID":"5544f4b6-217a-4264-9ce4-98a39eb48b3f","Type":"ContainerStarted","Data":"9c05c86d8c24eaec59fdda31312eb84ff5ec441b8183741f6930921e3c624320"} Jul 10 22:48:05.736878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:05.736799 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:05.751133 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:05.751090 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" podStartSLOduration=1.685428853 podStartE2EDuration="4.75107545s" totalImagesPullingTime="3.065646597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:48:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:48:05.749886346 +0000 UTC m=+344.387312020" watchObservedRunningTime="2026-07-10 22:48:05.75107545 +0000 UTC m=+344.388501125" Jul 10 22:48:05.767224 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:05.767178 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" podStartSLOduration=1.485385428 podStartE2EDuration="3.767164416s" totalImagesPullingTime="2.281778988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:48:05.766489122 +0000 UTC m=+344.403914797" watchObservedRunningTime="2026-07-10 22:48:05.767164416 +0000 UTC m=+344.404590091" Jul 10 22:48:10.761341 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.761301 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:10.822679 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.822647 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b2mll"] Jul 10 22:48:10.826066 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.826050 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:10.828602 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.828576 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 22:48:10.828829 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.828804 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tcjxn\"" Jul 10 22:48:10.840623 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.840599 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8"] Jul 10 22:48:10.843915 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.843895 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b2mll"] Jul 10 22:48:10.844022 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.843972 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:10.846557 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.846537 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7rmkb\"" Jul 10 22:48:10.846973 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.846958 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 22:48:10.860861 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.860839 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8"] Jul 10 22:48:10.864773 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.864745 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:10.958203 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:10.958169 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:11.000404 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.000373 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d4a83cb-1fbf-4852-8cc2-9a91972f4186-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ks4l8\" (UID: \"3d4a83cb-1fbf-4852-8cc2-9a91972f4186\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.000532 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.000486 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c-cert\") pod \"kserve-controller-manager-d6f4b65b8-b2mll\" (UID: \"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.000532 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.000517 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x82m9\" (UniqueName: \"kubernetes.io/projected/3d4a83cb-1fbf-4852-8cc2-9a91972f4186-kube-api-access-x82m9\") pod \"llmisvc-controller-manager-777c6bd7d5-ks4l8\" (UID: \"3d4a83cb-1fbf-4852-8cc2-9a91972f4186\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.000714 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.000546 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7jtw\" (UniqueName: \"kubernetes.io/projected/6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c-kube-api-access-t7jtw\") pod \"kserve-controller-manager-d6f4b65b8-b2mll\" (UID: \"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.101778 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.101711 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d4a83cb-1fbf-4852-8cc2-9a91972f4186-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ks4l8\" (UID: \"3d4a83cb-1fbf-4852-8cc2-9a91972f4186\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.101778 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.101771 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c-cert\") pod \"kserve-controller-manager-d6f4b65b8-b2mll\" (UID: \"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.101929 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.101803 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x82m9\" (UniqueName: \"kubernetes.io/projected/3d4a83cb-1fbf-4852-8cc2-9a91972f4186-kube-api-access-x82m9\") pod \"llmisvc-controller-manager-777c6bd7d5-ks4l8\" (UID: \"3d4a83cb-1fbf-4852-8cc2-9a91972f4186\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.101929 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.101842 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7jtw\" (UniqueName: \"kubernetes.io/projected/6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c-kube-api-access-t7jtw\") pod \"kserve-controller-manager-d6f4b65b8-b2mll\" (UID: \"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.104155 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.104129 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c-cert\") pod \"kserve-controller-manager-d6f4b65b8-b2mll\" (UID: \"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.104273 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.104197 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d4a83cb-1fbf-4852-8cc2-9a91972f4186-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ks4l8\" (UID: \"3d4a83cb-1fbf-4852-8cc2-9a91972f4186\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.125473 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.125452 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x82m9\" (UniqueName: \"kubernetes.io/projected/3d4a83cb-1fbf-4852-8cc2-9a91972f4186-kube-api-access-x82m9\") pod \"llmisvc-controller-manager-777c6bd7d5-ks4l8\" (UID: \"3d4a83cb-1fbf-4852-8cc2-9a91972f4186\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.138431 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.138411 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7jtw\" (UniqueName: \"kubernetes.io/projected/6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c-kube-api-access-t7jtw\") pod \"kserve-controller-manager-d6f4b65b8-b2mll\" (UID: \"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.154231 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.154193 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:11.286651 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.286628 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8"] Jul 10 22:48:11.288918 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:48:11.288891 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3d4a83cb_1fbf_4852_8cc2_9a91972f4186.slice/crio-20a381570877bc4175c480c597687e33cfa5b59c4be286030065a4939d8b6bfe WatchSource:0}: Error finding container 20a381570877bc4175c480c597687e33cfa5b59c4be286030065a4939d8b6bfe: Status 404 returned error can't find the container with id 20a381570877bc4175c480c597687e33cfa5b59c4be286030065a4939d8b6bfe Jul 10 22:48:11.362681 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.362590 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:11.378875 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.378853 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx"] Jul 10 22:48:11.384127 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.384107 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.386763 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.386740 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 22:48:11.386885 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.386867 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lfrsl\"" Jul 10 22:48:11.386951 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.386872 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 22:48:11.395725 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.395705 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx"] Jul 10 22:48:11.436562 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.436535 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:11.505614 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.505587 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tmp\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.505742 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.505637 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tls-certs\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.505742 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.505664 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zcrn\" (UniqueName: \"kubernetes.io/projected/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-kube-api-access-7zcrn\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.587519 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.587486 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b2mll"] Jul 10 22:48:11.596977 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:48:11.596945 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0a9bc9_f645_4bbe_b8f0_c7e095a2d72c.slice/crio-fd160ee50f98a141d1146ec3ee2b20a1926039c1f378692e6ef4e01848a7d49a WatchSource:0}: Error finding container fd160ee50f98a141d1146ec3ee2b20a1926039c1f378692e6ef4e01848a7d49a: Status 404 returned error can't find the container with id fd160ee50f98a141d1146ec3ee2b20a1926039c1f378692e6ef4e01848a7d49a Jul 10 22:48:11.607008 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.606987 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zcrn\" (UniqueName: \"kubernetes.io/projected/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-kube-api-access-7zcrn\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.607093 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.607047 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tmp\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.607093 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.607077 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tls-certs\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.607166 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:48:11.607155 2551 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 22:48:11.607222 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:48:11.607199 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tls-certs podName:3ca65be0-4a38-41a7-b4b8-f2f44dbe7556 nodeName:}" failed. No retries permitted until 2026-07-10 22:48:12.107185016 +0000 UTC m=+350.744610670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tls-certs") pod "kube-auth-proxy-7454dcb6b8-kt2gx" (UID: "3ca65be0-4a38-41a7-b4b8-f2f44dbe7556") : secret "kube-auth-proxy-tls" not found Jul 10 22:48:11.609066 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.609047 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tmp\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.628801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.628747 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zcrn\" (UniqueName: \"kubernetes.io/projected/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-kube-api-access-7zcrn\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:11.745201 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.745167 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:11.764550 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.764517 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" event={"ID":"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c","Type":"ContainerStarted","Data":"fd160ee50f98a141d1146ec3ee2b20a1926039c1f378692e6ef4e01848a7d49a"} Jul 10 22:48:11.765820 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:11.765797 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" event={"ID":"3d4a83cb-1fbf-4852-8cc2-9a91972f4186","Type":"ContainerStarted","Data":"20a381570877bc4175c480c597687e33cfa5b59c4be286030065a4939d8b6bfe"} Jul 10 22:48:12.101370 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.101335 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:12.112485 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.112459 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tls-certs\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:12.115416 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.115390 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ca65be0-4a38-41a7-b4b8-f2f44dbe7556-tls-certs\") pod \"kube-auth-proxy-7454dcb6b8-kt2gx\" (UID: \"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556\") " pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:12.230801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.230767 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:12.295374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.295339 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" Jul 10 22:48:12.327531 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.327500 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:12.686021 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.685952 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx"] Jul 10 22:48:12.688532 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:48:12.688503 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca65be0_4a38_41a7_b4b8_f2f44dbe7556.slice/crio-087618205fc24d0f77aabd9d87db4639f5a4d7ea1c591c1388f9993666e501c9 WatchSource:0}: Error finding container 087618205fc24d0f77aabd9d87db4639f5a4d7ea1c591c1388f9993666e501c9: Status 404 returned error can't find the container with id 087618205fc24d0f77aabd9d87db4639f5a4d7ea1c591c1388f9993666e501c9 Jul 10 22:48:12.772686 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:12.772651 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" event={"ID":"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556","Type":"ContainerStarted","Data":"087618205fc24d0f77aabd9d87db4639f5a4d7ea1c591c1388f9993666e501c9"} Jul 10 22:48:13.680335 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:13.680203 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:13.867990 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:13.867909 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:15.283993 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.283953 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:15.787884 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.787846 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" event={"ID":"3ca65be0-4a38-41a7-b4b8-f2f44dbe7556","Type":"ContainerStarted","Data":"9cfd7aa02efe7560239db9abe3a4cf6f8b184a4ea685792afe977c73e3017e06"} Jul 10 22:48:15.789309 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.789279 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" event={"ID":"3d4a83cb-1fbf-4852-8cc2-9a91972f4186","Type":"ContainerStarted","Data":"3b6b1636e45fa82b9bcc822af4b46c262b809e09c111a18cd2b103ee8a91d976"} Jul 10 22:48:15.789439 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.789340 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:15.790802 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.790778 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" event={"ID":"6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c","Type":"ContainerStarted","Data":"b55a5636448bd4dd3537a05d3300a0393344965e687f955687e50b24a26f234c"} Jul 10 22:48:15.790889 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.790830 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:15.815165 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.815118 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7454dcb6b8-kt2gx" podStartSLOduration=2.047043663 podStartE2EDuration="4.815103143s" totalImagesPullingTime="2.76805948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:48:15.813843889 +0000 UTC m=+354.451269602" watchObservedRunningTime="2026-07-10 22:48:15.815103143 +0000 UTC m=+354.452528855" Jul 10 22:48:15.837372 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.837291 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" podStartSLOduration=1.6748335490000001 podStartE2EDuration="5.837276401s" totalImagesPullingTime="4.162442852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:48:15.836021936 +0000 UTC m=+354.473447611" watchObservedRunningTime="2026-07-10 22:48:15.837276401 +0000 UTC m=+354.474702078" Jul 10 22:48:15.854908 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:15.854872 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" podStartSLOduration=2.00328802 podStartE2EDuration="5.854859707s" totalImagesPullingTime="3.851571687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:48:15.853473744 +0000 UTC m=+354.490899418" watchObservedRunningTime="2026-07-10 22:48:15.854859707 +0000 UTC m=+354.492285380" Jul 10 22:48:16.745957 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:16.745927 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-2hp5d" Jul 10 22:48:16.765714 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:16.765683 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hb647" Jul 10 22:48:17.730071 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:17.730037 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:17.858373 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:17.858345 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:18.377195 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.377159 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6"] Jul 10 22:48:18.381169 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.381148 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.383678 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.383656 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bpk4b\"" Jul 10 22:48:18.383843 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.383665 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 22:48:18.395793 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.395767 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6"] Jul 10 22:48:18.415873 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.415845 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:18.469052 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469029 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469151 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469063 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469151 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469087 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469151 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469149 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469176 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469206 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469258 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469278 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk9j5\" (UniqueName: \"kubernetes.io/projected/4e903ff9-e362-40e6-838e-084fbf1c07d8-kube-api-access-jk9j5\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.469351 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.469311 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e903ff9-e362-40e6-838e-084fbf1c07d8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570535 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570507 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e903ff9-e362-40e6-838e-084fbf1c07d8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570638 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570559 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570638 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570593 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570638 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570620 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570670 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570695 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570726 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570762 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.570982 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.570790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk9j5\" (UniqueName: \"kubernetes.io/projected/4e903ff9-e362-40e6-838e-084fbf1c07d8-kube-api-access-jk9j5\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.571162 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.571140 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e903ff9-e362-40e6-838e-084fbf1c07d8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.571492 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.571468 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.571672 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.571650 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.571794 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.571661 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.571881 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.571684 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.573839 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.573819 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.574013 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.573993 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.579445 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.579424 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk9j5\" (UniqueName: \"kubernetes.io/projected/4e903ff9-e362-40e6-838e-084fbf1c07d8-kube-api-access-jk9j5\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.579636 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.579616 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e903ff9-e362-40e6-838e-084fbf1c07d8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678p8qr6\" (UID: \"4e903ff9-e362-40e6-838e-084fbf1c07d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.611942 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.611912 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:18.675940 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.675881 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:18.693715 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.693693 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:18.802840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.802797 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:18.837115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.837089 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6"] Jul 10 22:48:18.840732 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:48:18.840706 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e903ff9_e362_40e6_838e_084fbf1c07d8.slice/crio-fd64491900d47b5e675c757c066f77fdf5e781b71323dc83136db6055770deeb WatchSource:0}: Error finding container fd64491900d47b5e675c757c066f77fdf5e781b71323dc83136db6055770deeb: Status 404 returned error can't find the container with id fd64491900d47b5e675c757c066f77fdf5e781b71323dc83136db6055770deeb Jul 10 22:48:18.913385 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:18.913355 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:48:19.828767 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:19.828736 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" event={"ID":"4e903ff9-e362-40e6-838e-084fbf1c07d8","Type":"ContainerStarted","Data":"fd64491900d47b5e675c757c066f77fdf5e781b71323dc83136db6055770deeb"} Jul 10 22:48:21.321104 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:21.321071 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:48:21.321407 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:21.321135 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:48:21.321407 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:21.321162 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:48:21.840601 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:21.840552 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" event={"ID":"4e903ff9-e362-40e6-838e-084fbf1c07d8","Type":"ContainerStarted","Data":"d35f1cde5b983a99c7b74e64cd1761f147faaf6f6a5933ca73dcc15a6ba6a08f"} Jul 10 22:48:21.863397 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:21.863348 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" podStartSLOduration=1.3851110979999999 podStartE2EDuration="3.863333285s" totalImagesPullingTime="2.478222187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:48:21.862435557 +0000 UTC m=+360.499861255" watchObservedRunningTime="2026-07-10 22:48:21.863333285 +0000 UTC m=+360.500758960" Jul 10 22:48:22.694761 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:22.694729 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:22.699520 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:22.699501 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:22.845059 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:22.845036 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:22.845788 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:22.845771 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678p8qr6" Jul 10 22:48:46.817776 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:46.817746 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ks4l8" Jul 10 22:48:46.820224 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:46.820193 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b2mll" Jul 10 22:48:49.635062 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:48:49.635032 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:01.785379 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.785342 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:01.886225 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.886170 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:01.909057 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.909025 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxdrw"] Jul 10 22:49:01.921069 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.921040 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:01.924162 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.923906 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 22:49:01.924162 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.923950 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6wq6n\"" Jul 10 22:49:01.924606 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.924585 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxdrw"] Jul 10 22:49:01.925134 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.925104 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 22:49:01.975420 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.975393 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv8wk\" (UniqueName: \"kubernetes.io/projected/a0b9e143-ba00-483c-aba2-ec998f7ad55a-kube-api-access-jv8wk\") pod \"kuadrant-operator-catalog-vxdrw\" (UID: \"a0b9e143-ba00-483c-aba2-ec998f7ad55a\") " pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:01.981736 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:01.981700 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:02.076723 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.076652 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv8wk\" (UniqueName: \"kubernetes.io/projected/a0b9e143-ba00-483c-aba2-ec998f7ad55a-kube-api-access-jv8wk\") pod \"kuadrant-operator-catalog-vxdrw\" (UID: \"a0b9e143-ba00-483c-aba2-ec998f7ad55a\") " pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:02.085450 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.085427 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv8wk\" (UniqueName: \"kubernetes.io/projected/a0b9e143-ba00-483c-aba2-ec998f7ad55a-kube-api-access-jv8wk\") pod \"kuadrant-operator-catalog-vxdrw\" (UID: \"a0b9e143-ba00-483c-aba2-ec998f7ad55a\") " pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:02.233448 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.233407 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:02.271524 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.271447 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxdrw"] Jul 10 22:49:02.366289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.366237 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxdrw"] Jul 10 22:49:02.368621 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:49:02.368584 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b9e143_ba00_483c_aba2_ec998f7ad55a.slice/crio-e4a3b7a2213741eeac9a8d753038379482ea0d63261fa5422107e5437f5b7740 WatchSource:0}: Error finding container e4a3b7a2213741eeac9a8d753038379482ea0d63261fa5422107e5437f5b7740: Status 404 returned error can't find the container with id e4a3b7a2213741eeac9a8d753038379482ea0d63261fa5422107e5437f5b7740 Jul 10 22:49:02.483660 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.483634 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7cmph"] Jul 10 22:49:02.487799 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.487784 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:02.495733 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.495707 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7cmph"] Jul 10 22:49:02.580654 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.580631 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmr6n\" (UniqueName: \"kubernetes.io/projected/866b30b1-0a41-4b39-addf-4900a909f1d3-kube-api-access-cmr6n\") pod \"kuadrant-operator-catalog-7cmph\" (UID: \"866b30b1-0a41-4b39-addf-4900a909f1d3\") " pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:02.681985 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.681926 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmr6n\" (UniqueName: \"kubernetes.io/projected/866b30b1-0a41-4b39-addf-4900a909f1d3-kube-api-access-cmr6n\") pod \"kuadrant-operator-catalog-7cmph\" (UID: \"866b30b1-0a41-4b39-addf-4900a909f1d3\") " pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:02.689987 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.689963 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmr6n\" (UniqueName: \"kubernetes.io/projected/866b30b1-0a41-4b39-addf-4900a909f1d3-kube-api-access-cmr6n\") pod \"kuadrant-operator-catalog-7cmph\" (UID: \"866b30b1-0a41-4b39-addf-4900a909f1d3\") " pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:02.797378 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.797353 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:02.927540 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:02.927491 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7cmph"] Jul 10 22:49:02.929597 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:49:02.929574 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866b30b1_0a41_4b39_addf_4900a909f1d3.slice/crio-8c43516ed9041fef906357025fa7572dbef16cec11f684eea2036318d1d5d9dd WatchSource:0}: Error finding container 8c43516ed9041fef906357025fa7572dbef16cec11f684eea2036318d1d5d9dd: Status 404 returned error can't find the container with id 8c43516ed9041fef906357025fa7572dbef16cec11f684eea2036318d1d5d9dd Jul 10 22:49:03.013774 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:03.013749 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" event={"ID":"866b30b1-0a41-4b39-addf-4900a909f1d3","Type":"ContainerStarted","Data":"8c43516ed9041fef906357025fa7572dbef16cec11f684eea2036318d1d5d9dd"} Jul 10 22:49:03.014916 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:03.014898 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" event={"ID":"a0b9e143-ba00-483c-aba2-ec998f7ad55a","Type":"ContainerStarted","Data":"e4a3b7a2213741eeac9a8d753038379482ea0d63261fa5422107e5437f5b7740"} Jul 10 22:49:05.031556 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.031518 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" event={"ID":"866b30b1-0a41-4b39-addf-4900a909f1d3","Type":"ContainerStarted","Data":"aa6146de29d0d6918cb22bedc5bc241a0a9f9c8d17b9bf24f82d4c172ee93257"} Jul 10 22:49:05.033098 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.033073 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" event={"ID":"a0b9e143-ba00-483c-aba2-ec998f7ad55a","Type":"ContainerStarted","Data":"ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6"} Jul 10 22:49:05.033230 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.033164 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" podUID="a0b9e143-ba00-483c-aba2-ec998f7ad55a" containerName="registry-server" containerID="cri-o://ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6" gracePeriod=2 Jul 10 22:49:05.047781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.047736 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" podStartSLOduration=1.279322433 podStartE2EDuration="3.047723312s" totalImagesPullingTime="1.768400879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:49:05.046386734 +0000 UTC m=+403.683812410" watchObservedRunningTime="2026-07-10 22:49:05.047723312 +0000 UTC m=+403.685148987" Jul 10 22:49:05.061762 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.061726 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" podStartSLOduration=1.735649563 podStartE2EDuration="4.061714448s" totalImagesPullingTime="2.326064885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:49:05.060940443 +0000 UTC m=+403.698366118" watchObservedRunningTime="2026-07-10 22:49:05.061714448 +0000 UTC m=+403.699140127" Jul 10 22:49:05.276179 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.276153 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:05.407030 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.406977 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv8wk\" (UniqueName: \"kubernetes.io/projected/a0b9e143-ba00-483c-aba2-ec998f7ad55a-kube-api-access-jv8wk\") pod \"a0b9e143-ba00-483c-aba2-ec998f7ad55a\" (UID: \"a0b9e143-ba00-483c-aba2-ec998f7ad55a\") " Jul 10 22:49:05.408988 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.408958 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b9e143-ba00-483c-aba2-ec998f7ad55a-kube-api-access-jv8wk" (OuterVolumeSpecName: "kube-api-access-jv8wk") pod "a0b9e143-ba00-483c-aba2-ec998f7ad55a" (UID: "a0b9e143-ba00-483c-aba2-ec998f7ad55a"). InnerVolumeSpecName "kube-api-access-jv8wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:49:05.507950 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:05.507927 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv8wk\" (UniqueName: \"kubernetes.io/projected/a0b9e143-ba00-483c-aba2-ec998f7ad55a-kube-api-access-jv8wk\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:49:06.038626 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.038599 2551 generic.go:356] "Generic (PLEG): container finished" podID="a0b9e143-ba00-483c-aba2-ec998f7ad55a" containerID="ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6" exitCode=0 Jul 10 22:49:06.038948 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.038807 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" Jul 10 22:49:06.040702 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.040679 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" event={"ID":"a0b9e143-ba00-483c-aba2-ec998f7ad55a","Type":"ContainerDied","Data":"ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6"} Jul 10 22:49:06.040803 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.040717 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxdrw" event={"ID":"a0b9e143-ba00-483c-aba2-ec998f7ad55a","Type":"ContainerDied","Data":"e4a3b7a2213741eeac9a8d753038379482ea0d63261fa5422107e5437f5b7740"} Jul 10 22:49:06.040803 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.040738 2551 scope.go:117] "RemoveContainer" containerID="ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6" Jul 10 22:49:06.050277 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.050261 2551 scope.go:117] "RemoveContainer" containerID="ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6" Jul 10 22:49:06.050485 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:49:06.050469 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6\": container with ID starting with ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6 not found: ID does not exist" containerID="ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6" Jul 10 22:49:06.050537 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.050490 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6"} err="failed to get container status \"ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6\": rpc error: code = NotFound desc = could not find container \"ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6\": container with ID starting with ad8b5209f9c350f90082d6fcc65af401f98a357ebe34daf1b876029648dc26b6 not found: ID does not exist" Jul 10 22:49:06.064417 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.064392 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxdrw"] Jul 10 22:49:06.069160 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:06.069140 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxdrw"] Jul 10 22:49:08.041071 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:08.041038 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0b9e143-ba00-483c-aba2-ec998f7ad55a" path="/var/lib/kubelet/pods/a0b9e143-ba00-483c-aba2-ec998f7ad55a/volumes" Jul 10 22:49:12.797731 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:12.797699 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:12.798127 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:12.797750 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:12.819648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:12.819625 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:13.094842 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:13.094768 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-7cmph" Jul 10 22:49:30.878756 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:30.878717 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:31.682521 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:31.682482 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:32.701939 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:32.701907 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:32.806066 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:32.806036 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:34.763037 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.763006 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc"] Jul 10 22:49:34.763438 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.763378 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b9e143-ba00-483c-aba2-ec998f7ad55a" containerName="registry-server" Jul 10 22:49:34.763438 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.763390 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b9e143-ba00-483c-aba2-ec998f7ad55a" containerName="registry-server" Jul 10 22:49:34.763509 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.763446 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0b9e143-ba00-483c-aba2-ec998f7ad55a" containerName="registry-server" Jul 10 22:49:34.767581 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.767564 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:34.769995 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.769977 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lgx5s\"" Jul 10 22:49:34.785554 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.785527 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc"] Jul 10 22:49:34.917595 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:34.917573 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz6ql\" (UniqueName: \"kubernetes.io/projected/073c116b-ef42-4331-a3f9-fdc7bea364fa-kube-api-access-vz6ql\") pod \"authorino-operator-75cc8d5cb7-nqcsc\" (UID: \"073c116b-ef42-4331-a3f9-fdc7bea364fa\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:35.018906 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:35.018852 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz6ql\" (UniqueName: \"kubernetes.io/projected/073c116b-ef42-4331-a3f9-fdc7bea364fa-kube-api-access-vz6ql\") pod \"authorino-operator-75cc8d5cb7-nqcsc\" (UID: \"073c116b-ef42-4331-a3f9-fdc7bea364fa\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:35.026922 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:35.026903 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz6ql\" (UniqueName: \"kubernetes.io/projected/073c116b-ef42-4331-a3f9-fdc7bea364fa-kube-api-access-vz6ql\") pod \"authorino-operator-75cc8d5cb7-nqcsc\" (UID: \"073c116b-ef42-4331-a3f9-fdc7bea364fa\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:35.077379 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:35.077359 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:35.209489 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:35.209427 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc"] Jul 10 22:49:35.212171 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:49:35.212146 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073c116b_ef42_4331_a3f9_fdc7bea364fa.slice/crio-320fb7f21381d8ff834048a8667fbaa2740989e08375fb1eb7623573d0fb6d7e WatchSource:0}: Error finding container 320fb7f21381d8ff834048a8667fbaa2740989e08375fb1eb7623573d0fb6d7e: Status 404 returned error can't find the container with id 320fb7f21381d8ff834048a8667fbaa2740989e08375fb1eb7623573d0fb6d7e Jul 10 22:49:36.138859 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:36.138816 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:36.172713 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:36.172681 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" event={"ID":"073c116b-ef42-4331-a3f9-fdc7bea364fa","Type":"ContainerStarted","Data":"320fb7f21381d8ff834048a8667fbaa2740989e08375fb1eb7623573d0fb6d7e"} Jul 10 22:49:38.182859 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.182813 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" event={"ID":"073c116b-ef42-4331-a3f9-fdc7bea364fa","Type":"ContainerStarted","Data":"b138c1c6f1a785f24e6e34966c2a31bf4d48364e65611782268c71d4dbd98d98"} Jul 10 22:49:38.183230 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.182880 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:38.197496 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.197469 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n"] Jul 10 22:49:38.200677 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.200639 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" podStartSLOduration=1.976551755 podStartE2EDuration="4.200626866s" totalImagesPullingTime="2.224075111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:49:38.19904027 +0000 UTC m=+436.836465946" watchObservedRunningTime="2026-07-10 22:49:38.200626866 +0000 UTC m=+436.838052541" Jul 10 22:49:38.204915 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.204898 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:38.207271 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.207252 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ztl56\"" Jul 10 22:49:38.214101 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.214076 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n"] Jul 10 22:49:38.346859 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.346833 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhctl\" (UniqueName: \"kubernetes.io/projected/937cf36f-f5c3-4c1c-a345-e878a119a6d7-kube-api-access-jhctl\") pod \"limitador-operator-controller-manager-859d7b78bd-v988n\" (UID: \"937cf36f-f5c3-4c1c-a345-e878a119a6d7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:38.447426 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.447361 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhctl\" (UniqueName: \"kubernetes.io/projected/937cf36f-f5c3-4c1c-a345-e878a119a6d7-kube-api-access-jhctl\") pod \"limitador-operator-controller-manager-859d7b78bd-v988n\" (UID: \"937cf36f-f5c3-4c1c-a345-e878a119a6d7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:38.457405 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.457383 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhctl\" (UniqueName: \"kubernetes.io/projected/937cf36f-f5c3-4c1c-a345-e878a119a6d7-kube-api-access-jhctl\") pod \"limitador-operator-controller-manager-859d7b78bd-v988n\" (UID: \"937cf36f-f5c3-4c1c-a345-e878a119a6d7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:38.516701 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.516674 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:38.645950 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.645928 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n"] Jul 10 22:49:38.648092 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:49:38.648063 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937cf36f_f5c3_4c1c_a345_e878a119a6d7.slice/crio-b1a84925de3cc66fba10e2bff37a16a09ad02696ef5b2dac2ed1b52581e4824e WatchSource:0}: Error finding container b1a84925de3cc66fba10e2bff37a16a09ad02696ef5b2dac2ed1b52581e4824e: Status 404 returned error can't find the container with id b1a84925de3cc66fba10e2bff37a16a09ad02696ef5b2dac2ed1b52581e4824e Jul 10 22:49:38.915138 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:38.915099 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:39.192322 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:39.192246 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" event={"ID":"937cf36f-f5c3-4c1c-a345-e878a119a6d7","Type":"ContainerStarted","Data":"b1a84925de3cc66fba10e2bff37a16a09ad02696ef5b2dac2ed1b52581e4824e"} Jul 10 22:49:39.805629 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:39.805596 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:41.130904 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:41.130879 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:41.202748 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:41.202721 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" event={"ID":"937cf36f-f5c3-4c1c-a345-e878a119a6d7","Type":"ContainerStarted","Data":"3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c"} Jul 10 22:49:41.202878 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:41.202842 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:41.220451 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:41.220406 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" podStartSLOduration=1.53197522 podStartE2EDuration="3.220392902s" totalImagesPullingTime="1.688417682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:49:41.218754565 +0000 UTC m=+439.856180254" watchObservedRunningTime="2026-07-10 22:49:41.220392902 +0000 UTC m=+439.857818577" Jul 10 22:49:42.688018 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:42.687987 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:43.320760 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:43.320731 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:45.687653 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:45.687622 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:45.788664 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:45.788631 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:46.118337 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:46.118309 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:46.219803 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:46.219774 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:49.194526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:49.194498 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nqcsc" Jul 10 22:49:49.300935 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:49.300904 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:49.397471 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:49.397442 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:51.006614 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:51.006568 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:51.304947 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:51.304864 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:51.805905 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:51.805857 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:51.905547 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:51.905520 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:52.210067 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:52.209999 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:49:52.727528 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:52.727496 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:52.819274 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:52.819241 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:58.511150 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:58.511113 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:49:58.615051 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:49:58.615021 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:01.142708 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.142676 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z"] Jul 10 22:50:01.146151 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.146131 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.149783 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.149762 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tj8sl\"" Jul 10 22:50:01.164074 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.164050 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z"] Jul 10 22:50:01.211128 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.211100 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/778e2a7b-0cd1-4728-acfe-ac42c2153004-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.211289 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.211137 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mspfj\" (UniqueName: \"kubernetes.io/projected/778e2a7b-0cd1-4728-acfe-ac42c2153004-kube-api-access-mspfj\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.312268 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.312238 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/778e2a7b-0cd1-4728-acfe-ac42c2153004-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.312403 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.312272 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mspfj\" (UniqueName: \"kubernetes.io/projected/778e2a7b-0cd1-4728-acfe-ac42c2153004-kube-api-access-mspfj\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.312603 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.312585 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/778e2a7b-0cd1-4728-acfe-ac42c2153004-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.323835 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.323809 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mspfj\" (UniqueName: \"kubernetes.io/projected/778e2a7b-0cd1-4728-acfe-ac42c2153004-kube-api-access-mspfj\") pod \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.363354 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.363326 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z"] Jul 10 22:50:01.363610 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.363598 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:01.368607 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.368573 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z"] Jul 10 22:50:01.387315 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.387288 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg"] Jul 10 22:50:01.391442 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.391425 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.401399 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.401376 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg"] Jul 10 22:50:01.412727 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.412700 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78318201-330d-4d0e-a548-fe1a6bc20eab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-p6fdg\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.412831 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.412741 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q2sn\" (UniqueName: \"kubernetes.io/projected/78318201-330d-4d0e-a548-fe1a6bc20eab-kube-api-access-4q2sn\") pod \"kuadrant-operator-controller-manager-55467d9f7-p6fdg\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.425246 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.425222 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n"] Jul 10 22:50:01.425429 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.425408 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" containerName="manager" containerID="cri-o://3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c" gracePeriod=2 Jul 10 22:50:01.441139 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.441080 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n"] Jul 10 22:50:01.449124 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:01.449088 2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"937cf36f-f5c3-4c1c-a345-e878a119a6d7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-v988n\": pods \"limitador-operator-controller-manager-859d7b78bd-v988n\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:50:01.513447 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.513417 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78318201-330d-4d0e-a548-fe1a6bc20eab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-p6fdg\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.513599 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.513453 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q2sn\" (UniqueName: \"kubernetes.io/projected/78318201-330d-4d0e-a548-fe1a6bc20eab-kube-api-access-4q2sn\") pod \"kuadrant-operator-controller-manager-55467d9f7-p6fdg\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.513808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.513785 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78318201-330d-4d0e-a548-fe1a6bc20eab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-p6fdg\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.522869 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.522844 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q2sn\" (UniqueName: \"kubernetes.io/projected/78318201-330d-4d0e-a548-fe1a6bc20eab-kube-api-access-4q2sn\") pod \"kuadrant-operator-controller-manager-55467d9f7-p6fdg\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.657079 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.657056 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:50:01.659370 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:01.659343 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v988n\" is forbidden: User \"system:node:ip-10-0-138-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-188.ec2.internal' and this object" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:50:01.704468 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.704438 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:01.816315 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.816274 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhctl\" (UniqueName: \"kubernetes.io/projected/937cf36f-f5c3-4c1c-a345-e878a119a6d7-kube-api-access-jhctl\") pod \"937cf36f-f5c3-4c1c-a345-e878a119a6d7\" (UID: \"937cf36f-f5c3-4c1c-a345-e878a119a6d7\") " Jul 10 22:50:01.818693 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.818662 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937cf36f-f5c3-4c1c-a345-e878a119a6d7-kube-api-access-jhctl" (OuterVolumeSpecName: "kube-api-access-jhctl") pod "937cf36f-f5c3-4c1c-a345-e878a119a6d7" (UID: "937cf36f-f5c3-4c1c-a345-e878a119a6d7"). InnerVolumeSpecName "kube-api-access-jhctl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:50:01.842268 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.842178 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg"] Jul 10 22:50:01.844806 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:50:01.844786 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78318201_330d_4d0e_a548_fe1a6bc20eab.slice/crio-17b2dc1d953a1c27927db011892de3812221443e3e1105d09556a8eef237f0f0 WatchSource:0}: Error finding container 17b2dc1d953a1c27927db011892de3812221443e3e1105d09556a8eef237f0f0: Status 404 returned error can't find the container with id 17b2dc1d953a1c27927db011892de3812221443e3e1105d09556a8eef237f0f0 Jul 10 22:50:01.917384 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:01.917317 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhctl\" (UniqueName: \"kubernetes.io/projected/937cf36f-f5c3-4c1c-a345-e878a119a6d7-kube-api-access-jhctl\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:50:02.039536 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:02.039497 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v988n\" is forbidden: User \"system:node:ip-10-0-138-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-188.ec2.internal' and this object" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:50:02.041324 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.041300 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" path="/var/lib/kubelet/pods/937cf36f-f5c3-4c1c-a345-e878a119a6d7/volumes" Jul 10 22:50:02.295955 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.295920 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" event={"ID":"78318201-330d-4d0e-a548-fe1a6bc20eab","Type":"ContainerStarted","Data":"17b2dc1d953a1c27927db011892de3812221443e3e1105d09556a8eef237f0f0"} Jul 10 22:50:02.297183 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.297153 2551 generic.go:356] "Generic (PLEG): container finished" podID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" containerID="3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c" exitCode=0 Jul 10 22:50:02.297316 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.297197 2551 scope.go:117] "RemoveContainer" containerID="3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c" Jul 10 22:50:02.297316 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.297201 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v988n" Jul 10 22:50:02.305480 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.305464 2551 scope.go:117] "RemoveContainer" containerID="3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c" Jul 10 22:50:02.305734 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:02.305715 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c\": container with ID starting with 3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c not found: ID does not exist" containerID="3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c" Jul 10 22:50:02.305785 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.305743 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c"} err="failed to get container status \"3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c\": rpc error: code = NotFound desc = could not find container \"3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c\": container with ID starting with 3af1a08c7c64f26e55f30ef694a34f5dfc39326c486e52d20fed9c50c92e5d4c not found: ID does not exist" Jul 10 22:50:02.918739 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:02.918697 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:03.015418 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:03.015391 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:03.906843 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:03.906793 2551 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-m8b2z_kuadrant-system_778e2a7b-0cd1-4728-acfe-ac42c2153004_0(fb54c0ff6c1926aff927b61e993737068659018411303086bfaf173d76722e5d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-m8b2z to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 10 22:50:03.907124 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:03.906859 2551 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-m8b2z_kuadrant-system_778e2a7b-0cd1-4728-acfe-ac42c2153004_0(fb54c0ff6c1926aff927b61e993737068659018411303086bfaf173d76722e5d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-m8b2z to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:04.214985 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.214900 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:04.303967 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.303914 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:04.309357 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.309328 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:04.319042 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.319020 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:04.321542 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:04.321515 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" is forbidden: User \"system:node:ip-10-0-138-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-188.ec2.internal' and this object" podUID="778e2a7b-0cd1-4728-acfe-ac42c2153004" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:04.335805 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.335782 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/778e2a7b-0cd1-4728-acfe-ac42c2153004-extensions-socket-volume\") pod \"778e2a7b-0cd1-4728-acfe-ac42c2153004\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " Jul 10 22:50:04.335927 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.335839 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mspfj\" (UniqueName: \"kubernetes.io/projected/778e2a7b-0cd1-4728-acfe-ac42c2153004-kube-api-access-mspfj\") pod \"778e2a7b-0cd1-4728-acfe-ac42c2153004\" (UID: \"778e2a7b-0cd1-4728-acfe-ac42c2153004\") " Jul 10 22:50:04.336071 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.336048 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/778e2a7b-0cd1-4728-acfe-ac42c2153004-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "778e2a7b-0cd1-4728-acfe-ac42c2153004" (UID: "778e2a7b-0cd1-4728-acfe-ac42c2153004"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 22:50:04.337981 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.337957 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778e2a7b-0cd1-4728-acfe-ac42c2153004-kube-api-access-mspfj" (OuterVolumeSpecName: "kube-api-access-mspfj") pod "778e2a7b-0cd1-4728-acfe-ac42c2153004" (UID: "778e2a7b-0cd1-4728-acfe-ac42c2153004"). InnerVolumeSpecName "kube-api-access-mspfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:50:04.436447 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.436420 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/778e2a7b-0cd1-4728-acfe-ac42c2153004-extensions-socket-volume\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:50:04.436447 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.436449 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mspfj\" (UniqueName: \"kubernetes.io/projected/778e2a7b-0cd1-4728-acfe-ac42c2153004-kube-api-access-mspfj\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:50:04.781769 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.781726 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:04.894556 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:04.894514 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:05.316325 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:05.316287 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:05.322654 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:05.322616 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" is forbidden: User \"system:node:ip-10-0-138-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-188.ec2.internal' and this object" podUID="778e2a7b-0cd1-4728-acfe-ac42c2153004" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:05.326767 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:05.326739 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-m8b2z\" is forbidden: User \"system:node:ip-10-0-138-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-188.ec2.internal' and this object" podUID="778e2a7b-0cd1-4728-acfe-ac42c2153004" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m8b2z" Jul 10 22:50:06.042304 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:06.042271 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="778e2a7b-0cd1-4728-acfe-ac42c2153004" path="/var/lib/kubelet/pods/778e2a7b-0cd1-4728-acfe-ac42c2153004/volumes" Jul 10 22:50:06.321892 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:06.321807 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" event={"ID":"78318201-330d-4d0e-a548-fe1a6bc20eab","Type":"ContainerStarted","Data":"7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9"} Jul 10 22:50:06.322272 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:06.321953 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:06.344853 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:06.344805 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" podStartSLOduration=1.121740485 podStartE2EDuration="5.344792107s" totalImagesPullingTime="4.223051622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:50:06.343651998 +0000 UTC m=+464.981077711" watchObservedRunningTime="2026-07-10 22:50:06.344792107 +0000 UTC m=+464.982217782" Jul 10 22:50:08.205456 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:08.205417 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:08.306700 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:08.306666 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:11.605766 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:11.605732 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:11.706051 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:11.706011 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:17.328452 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:17.328420 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:17.396765 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:17.396733 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:17.494505 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:17.494475 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:29.761170 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:29.761138 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg"] Jul 10 22:50:29.761659 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:29.761350 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" podUID="78318201-330d-4d0e-a548-fe1a6bc20eab" containerName="manager" containerID="cri-o://7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9" gracePeriod=10 Jul 10 22:50:30.009133 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.009104 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:30.123981 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.123918 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78318201-330d-4d0e-a548-fe1a6bc20eab-extensions-socket-volume\") pod \"78318201-330d-4d0e-a548-fe1a6bc20eab\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " Jul 10 22:50:30.123981 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.123946 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q2sn\" (UniqueName: \"kubernetes.io/projected/78318201-330d-4d0e-a548-fe1a6bc20eab-kube-api-access-4q2sn\") pod \"78318201-330d-4d0e-a548-fe1a6bc20eab\" (UID: \"78318201-330d-4d0e-a548-fe1a6bc20eab\") " Jul 10 22:50:30.124290 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.124268 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78318201-330d-4d0e-a548-fe1a6bc20eab-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "78318201-330d-4d0e-a548-fe1a6bc20eab" (UID: "78318201-330d-4d0e-a548-fe1a6bc20eab"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 22:50:30.125989 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.125966 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78318201-330d-4d0e-a548-fe1a6bc20eab-kube-api-access-4q2sn" (OuterVolumeSpecName: "kube-api-access-4q2sn") pod "78318201-330d-4d0e-a548-fe1a6bc20eab" (UID: "78318201-330d-4d0e-a548-fe1a6bc20eab"). InnerVolumeSpecName "kube-api-access-4q2sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:50:30.224720 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.224694 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78318201-330d-4d0e-a548-fe1a6bc20eab-extensions-socket-volume\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:50:30.224720 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.224719 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q2sn\" (UniqueName: \"kubernetes.io/projected/78318201-330d-4d0e-a548-fe1a6bc20eab-kube-api-access-4q2sn\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:50:30.437243 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.437167 2551 generic.go:356] "Generic (PLEG): container finished" podID="78318201-330d-4d0e-a548-fe1a6bc20eab" containerID="7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9" exitCode=0 Jul 10 22:50:30.437243 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.437234 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" Jul 10 22:50:30.437373 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.437238 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" event={"ID":"78318201-330d-4d0e-a548-fe1a6bc20eab","Type":"ContainerDied","Data":"7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9"} Jul 10 22:50:30.437373 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.437294 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg" event={"ID":"78318201-330d-4d0e-a548-fe1a6bc20eab","Type":"ContainerDied","Data":"17b2dc1d953a1c27927db011892de3812221443e3e1105d09556a8eef237f0f0"} Jul 10 22:50:30.437373 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.437315 2551 scope.go:117] "RemoveContainer" containerID="7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9" Jul 10 22:50:30.445912 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.445897 2551 scope.go:117] "RemoveContainer" containerID="7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9" Jul 10 22:50:30.446137 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:50:30.446121 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9\": container with ID starting with 7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9 not found: ID does not exist" containerID="7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9" Jul 10 22:50:30.446185 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.446144 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9"} err="failed to get container status \"7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9\": rpc error: code = NotFound desc = could not find container \"7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9\": container with ID starting with 7e2a03320323dd8dc77ce24a6eb0eaccb2746e413d7aa561364104dbc046f9d9 not found: ID does not exist" Jul 10 22:50:30.458427 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.458408 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg"] Jul 10 22:50:30.462426 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:30.462406 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p6fdg"] Jul 10 22:50:32.041504 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:32.041462 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78318201-330d-4d0e-a548-fe1a6bc20eab" path="/var/lib/kubelet/pods/78318201-330d-4d0e-a548-fe1a6bc20eab/volumes" Jul 10 22:50:33.716962 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.716931 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh"] Jul 10 22:50:33.717455 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.717440 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" containerName="manager" Jul 10 22:50:33.717504 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.717459 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" containerName="manager" Jul 10 22:50:33.717504 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.717471 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78318201-330d-4d0e-a548-fe1a6bc20eab" containerName="manager" Jul 10 22:50:33.717504 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.717481 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="78318201-330d-4d0e-a548-fe1a6bc20eab" containerName="manager" Jul 10 22:50:33.717594 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.717584 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="937cf36f-f5c3-4c1c-a345-e878a119a6d7" containerName="manager" Jul 10 22:50:33.717627 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.717595 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="78318201-330d-4d0e-a548-fe1a6bc20eab" containerName="manager" Jul 10 22:50:33.722576 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.722554 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.725098 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.725074 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zh7sh\"" Jul 10 22:50:33.733848 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.733816 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh"] Jul 10 22:50:33.746673 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.746595 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:33.855564 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855531 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855566 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab5b4c-6e90-4d13-99f4-bff938e21213-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855611 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855633 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gprpd\" (UniqueName: \"kubernetes.io/projected/27ab5b4c-6e90-4d13-99f4-bff938e21213-kube-api-access-gprpd\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855672 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855889 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855694 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855889 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855749 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855889 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855792 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.855889 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.855817 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.956622 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.956582 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.956622 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.956626 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab5b4c-6e90-4d13-99f4-bff938e21213-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.956801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.956672 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.956801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.956697 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gprpd\" (UniqueName: \"kubernetes.io/projected/27ab5b4c-6e90-4d13-99f4-bff938e21213-kube-api-access-gprpd\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.956801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.956738 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.956801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.956761 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957053 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957002 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957053 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957022 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957053 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957036 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957305 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957066 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957305 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957088 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957305 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957120 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957408 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957368 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.957928 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.957906 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27ab5b4c-6e90-4d13-99f4-bff938e21213-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.959193 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.959170 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.959477 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.959461 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.964149 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.964127 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27ab5b4c-6e90-4d13-99f4-bff938e21213-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:33.964332 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:33.964244 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gprpd\" (UniqueName: \"kubernetes.io/projected/27ab5b4c-6e90-4d13-99f4-bff938e21213-kube-api-access-gprpd\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lpbbh\" (UID: \"27ab5b4c-6e90-4d13-99f4-bff938e21213\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:34.033198 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.033176 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:34.149009 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.148974 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:34.174115 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.174091 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh"] Jul 10 22:50:34.177286 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:50:34.177264 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ab5b4c_6e90_4d13_99f4_bff938e21213.slice/crio-208712edb6ef8e367bc19018dbabdab3c441a6be55a9bc47fa9be50b526352c0 WatchSource:0}: Error finding container 208712edb6ef8e367bc19018dbabdab3c441a6be55a9bc47fa9be50b526352c0: Status 404 returned error can't find the container with id 208712edb6ef8e367bc19018dbabdab3c441a6be55a9bc47fa9be50b526352c0 Jul 10 22:50:34.179644 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.179614 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:50:34.179747 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.179679 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:50:34.179747 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.179709 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:50:34.465721 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.465638 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" event={"ID":"27ab5b4c-6e90-4d13-99f4-bff938e21213","Type":"ContainerStarted","Data":"e5779d17a0dc6bccea2105e701dc8c61d0f28ba2011818972d87a62d2e92c108"} Jul 10 22:50:34.465721 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.465687 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" event={"ID":"27ab5b4c-6e90-4d13-99f4-bff938e21213","Type":"ContainerStarted","Data":"208712edb6ef8e367bc19018dbabdab3c441a6be55a9bc47fa9be50b526352c0"} Jul 10 22:50:34.488785 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:34.488740 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" podStartSLOduration=1.488723352 podStartE2EDuration="1.488723352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:50:34.488350116 +0000 UTC m=+493.125775803" watchObservedRunningTime="2026-07-10 22:50:34.488723352 +0000 UTC m=+493.126149031" Jul 10 22:50:35.033880 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:35.033850 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:35.038387 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:35.038365 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:35.469992 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:35.469923 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:35.470740 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:35.470717 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:50:50.004387 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.004353 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:50:50.006726 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.006710 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.008392 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.008364 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:50.009378 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.009357 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 22:50:50.009454 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.009421 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-crb7z\"" Jul 10 22:50:50.015648 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.015626 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:50:50.081900 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.081869 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7de327d-428a-4ad5-808a-8af573f73137-config-file\") pod \"limitador-limitador-64c94fd64f-czmdp\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.082060 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.081992 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx99n\" (UniqueName: \"kubernetes.io/projected/a7de327d-428a-4ad5-808a-8af573f73137-kube-api-access-bx99n\") pod \"limitador-limitador-64c94fd64f-czmdp\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.097057 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.097030 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:50:50.110772 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.110742 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:50.182840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.182810 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx99n\" (UniqueName: \"kubernetes.io/projected/a7de327d-428a-4ad5-808a-8af573f73137-kube-api-access-bx99n\") pod \"limitador-limitador-64c94fd64f-czmdp\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.182989 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.182869 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7de327d-428a-4ad5-808a-8af573f73137-config-file\") pod \"limitador-limitador-64c94fd64f-czmdp\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.183488 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.183469 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7de327d-428a-4ad5-808a-8af573f73137-config-file\") pod \"limitador-limitador-64c94fd64f-czmdp\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.190800 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.190779 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx99n\" (UniqueName: \"kubernetes.io/projected/a7de327d-428a-4ad5-808a-8af573f73137-kube-api-access-bx99n\") pod \"limitador-limitador-64c94fd64f-czmdp\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.211714 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.211685 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:50.318782 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.318722 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:50.455349 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.455323 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:50:50.458135 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:50:50.458111 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7de327d_428a_4ad5_808a_8af573f73137.slice/crio-f54ef9970ead553547d33565849be66500fdff9331c5512d1b03eb3c140885cd WatchSource:0}: Error finding container f54ef9970ead553547d33565849be66500fdff9331c5512d1b03eb3c140885cd: Status 404 returned error can't find the container with id f54ef9970ead553547d33565849be66500fdff9331c5512d1b03eb3c140885cd Jul 10 22:50:50.483113 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.483082 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:50:50.485774 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.485756 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.494155 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.494129 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:50:50.508843 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.508820 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:50:50.537338 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.537310 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" event={"ID":"a7de327d-428a-4ad5-808a-8af573f73137","Type":"ContainerStarted","Data":"f54ef9970ead553547d33565849be66500fdff9331c5512d1b03eb3c140885cd"} Jul 10 22:50:50.586902 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.586844 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r97ws\" (UniqueName: \"kubernetes.io/projected/4c674571-05ec-49d0-990a-24bb1a026a69-kube-api-access-r97ws\") pod \"limitador-limitador-54d6d988d5-kpxpw\" (UID: \"4c674571-05ec-49d0-990a-24bb1a026a69\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.586993 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.586909 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c674571-05ec-49d0-990a-24bb1a026a69-config-file\") pod \"limitador-limitador-54d6d988d5-kpxpw\" (UID: \"4c674571-05ec-49d0-990a-24bb1a026a69\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.687374 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.687348 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r97ws\" (UniqueName: \"kubernetes.io/projected/4c674571-05ec-49d0-990a-24bb1a026a69-kube-api-access-r97ws\") pod \"limitador-limitador-54d6d988d5-kpxpw\" (UID: \"4c674571-05ec-49d0-990a-24bb1a026a69\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.687473 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.687445 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c674571-05ec-49d0-990a-24bb1a026a69-config-file\") pod \"limitador-limitador-54d6d988d5-kpxpw\" (UID: \"4c674571-05ec-49d0-990a-24bb1a026a69\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.688006 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.687983 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c674571-05ec-49d0-990a-24bb1a026a69-config-file\") pod \"limitador-limitador-54d6d988d5-kpxpw\" (UID: \"4c674571-05ec-49d0-990a-24bb1a026a69\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.695147 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.695127 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r97ws\" (UniqueName: \"kubernetes.io/projected/4c674571-05ec-49d0-990a-24bb1a026a69-kube-api-access-r97ws\") pod \"limitador-limitador-54d6d988d5-kpxpw\" (UID: \"4c674571-05ec-49d0-990a-24bb1a026a69\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.796874 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.796851 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:50.826685 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.826657 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:50.927758 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.927713 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:50.928293 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:50.928274 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:50:50.930063 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:50:50.930037 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c674571_05ec_49d0_990a_24bb1a026a69.slice/crio-2d6b452aba45cc93280a64a03e07912dfb949bc36b8d22da85236eeab2ddd878 WatchSource:0}: Error finding container 2d6b452aba45cc93280a64a03e07912dfb949bc36b8d22da85236eeab2ddd878: Status 404 returned error can't find the container with id 2d6b452aba45cc93280a64a03e07912dfb949bc36b8d22da85236eeab2ddd878 Jul 10 22:50:51.558029 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:51.557973 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" event={"ID":"4c674571-05ec-49d0-990a-24bb1a026a69","Type":"ContainerStarted","Data":"2d6b452aba45cc93280a64a03e07912dfb949bc36b8d22da85236eeab2ddd878"} Jul 10 22:50:53.548660 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.548632 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:53.568744 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.568718 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" event={"ID":"a7de327d-428a-4ad5-808a-8af573f73137","Type":"ContainerStarted","Data":"d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598"} Jul 10 22:50:53.568888 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.568868 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:50:53.570077 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.570058 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" event={"ID":"4c674571-05ec-49d0-990a-24bb1a026a69","Type":"ContainerStarted","Data":"a9832c942236f2cb15db7cc366b0d9d77c48388fbbb78e474de8e50260939612"} Jul 10 22:50:53.570177 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.570164 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:50:53.587101 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.587059 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" podStartSLOduration=1.666013772 podStartE2EDuration="4.587046273s" totalImagesPullingTime="2.921032501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:50:53.584900794 +0000 UTC m=+512.222326481" watchObservedRunningTime="2026-07-10 22:50:53.587046273 +0000 UTC m=+512.224471948" Jul 10 22:50:53.602156 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:53.602119 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" podStartSLOduration=1.160573423 podStartE2EDuration="3.602106553s" totalImagesPullingTime="2.44153313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:50:53.600979817 +0000 UTC m=+512.238405491" watchObservedRunningTime="2026-07-10 22:50:53.602106553 +0000 UTC m=+512.239532227" Jul 10 22:50:54.805646 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:54.805611 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:50:54.861245 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:50:54.861221 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:04.576126 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:04.576096 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:51:04.576570 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:04.576172 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-kpxpw" Jul 10 22:51:04.657857 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:04.657826 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:51:04.658033 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:04.657978 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" podUID="a7de327d-428a-4ad5-808a-8af573f73137" containerName="limitador" containerID="cri-o://d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598" gracePeriod=30 Jul 10 22:51:05.204854 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.204825 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:51:05.305021 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.304991 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx99n\" (UniqueName: \"kubernetes.io/projected/a7de327d-428a-4ad5-808a-8af573f73137-kube-api-access-bx99n\") pod \"a7de327d-428a-4ad5-808a-8af573f73137\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " Jul 10 22:51:05.305178 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.305045 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7de327d-428a-4ad5-808a-8af573f73137-config-file\") pod \"a7de327d-428a-4ad5-808a-8af573f73137\" (UID: \"a7de327d-428a-4ad5-808a-8af573f73137\") " Jul 10 22:51:05.305426 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.305403 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7de327d-428a-4ad5-808a-8af573f73137-config-file" (OuterVolumeSpecName: "config-file") pod "a7de327d-428a-4ad5-808a-8af573f73137" (UID: "a7de327d-428a-4ad5-808a-8af573f73137"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 22:51:05.307043 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.307022 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7de327d-428a-4ad5-808a-8af573f73137-kube-api-access-bx99n" (OuterVolumeSpecName: "kube-api-access-bx99n") pod "a7de327d-428a-4ad5-808a-8af573f73137" (UID: "a7de327d-428a-4ad5-808a-8af573f73137"). InnerVolumeSpecName "kube-api-access-bx99n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:51:05.412162 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.406739 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx99n\" (UniqueName: \"kubernetes.io/projected/a7de327d-428a-4ad5-808a-8af573f73137-kube-api-access-bx99n\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:51:05.412341 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.412193 2551 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7de327d-428a-4ad5-808a-8af573f73137-config-file\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:51:05.626680 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.626593 2551 generic.go:356] "Generic (PLEG): container finished" podID="a7de327d-428a-4ad5-808a-8af573f73137" containerID="d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598" exitCode=0 Jul 10 22:51:05.626680 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.626648 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" Jul 10 22:51:05.627099 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.626684 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" event={"ID":"a7de327d-428a-4ad5-808a-8af573f73137","Type":"ContainerDied","Data":"d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598"} Jul 10 22:51:05.627099 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.626761 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-czmdp" event={"ID":"a7de327d-428a-4ad5-808a-8af573f73137","Type":"ContainerDied","Data":"f54ef9970ead553547d33565849be66500fdff9331c5512d1b03eb3c140885cd"} Jul 10 22:51:05.627099 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.626783 2551 scope.go:117] "RemoveContainer" containerID="d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598" Jul 10 22:51:05.636795 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.636779 2551 scope.go:117] "RemoveContainer" containerID="d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598" Jul 10 22:51:05.637049 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:51:05.637028 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598\": container with ID starting with d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598 not found: ID does not exist" containerID="d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598" Jul 10 22:51:05.637114 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.637061 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598"} err="failed to get container status \"d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598\": rpc error: code = NotFound desc = could not find container \"d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598\": container with ID starting with d2ec46f1ed23b7500fde8da795e17a1873f4028fc852e4ee9bbe983e8ca1e598 not found: ID does not exist" Jul 10 22:51:05.650451 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.650431 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:51:05.655421 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:05.655401 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-czmdp"] Jul 10 22:51:06.042304 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:06.042267 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7de327d-428a-4ad5-808a-8af573f73137" path="/var/lib/kubelet/pods/a7de327d-428a-4ad5-808a-8af573f73137/volumes" Jul 10 22:51:06.931441 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:06.931408 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:07.029225 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.029181 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:07.341243 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.341197 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2"] Jul 10 22:51:07.341788 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.341771 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7de327d-428a-4ad5-808a-8af573f73137" containerName="limitador" Jul 10 22:51:07.341831 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.341792 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7de327d-428a-4ad5-808a-8af573f73137" containerName="limitador" Jul 10 22:51:07.341918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.341907 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7de327d-428a-4ad5-808a-8af573f73137" containerName="limitador" Jul 10 22:51:07.346787 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.346769 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.349741 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.349722 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 22:51:07.350413 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.350395 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 22:51:07.350530 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.350417 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 22:51:07.350530 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.350417 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-47b7m\"" Jul 10 22:51:07.353879 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.353857 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2"] Jul 10 22:51:07.374611 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.374577 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:07.531840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.531806 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e32f1d5d-f6fa-410f-ab26-b60faede8696-data\") pod \"postgres-69fb4594cf-6sbt2\" (UID: \"e32f1d5d-f6fa-410f-ab26-b60faede8696\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.532001 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.531907 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqd6x\" (UniqueName: \"kubernetes.io/projected/e32f1d5d-f6fa-410f-ab26-b60faede8696-kube-api-access-wqd6x\") pod \"postgres-69fb4594cf-6sbt2\" (UID: \"e32f1d5d-f6fa-410f-ab26-b60faede8696\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.632701 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.632618 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqd6x\" (UniqueName: \"kubernetes.io/projected/e32f1d5d-f6fa-410f-ab26-b60faede8696-kube-api-access-wqd6x\") pod \"postgres-69fb4594cf-6sbt2\" (UID: \"e32f1d5d-f6fa-410f-ab26-b60faede8696\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.632701 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.632680 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e32f1d5d-f6fa-410f-ab26-b60faede8696-data\") pod \"postgres-69fb4594cf-6sbt2\" (UID: \"e32f1d5d-f6fa-410f-ab26-b60faede8696\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.633021 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.633006 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e32f1d5d-f6fa-410f-ab26-b60faede8696-data\") pod \"postgres-69fb4594cf-6sbt2\" (UID: \"e32f1d5d-f6fa-410f-ab26-b60faede8696\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.641446 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.641419 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqd6x\" (UniqueName: \"kubernetes.io/projected/e32f1d5d-f6fa-410f-ab26-b60faede8696-kube-api-access-wqd6x\") pod \"postgres-69fb4594cf-6sbt2\" (UID: \"e32f1d5d-f6fa-410f-ab26-b60faede8696\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.658185 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.658161 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:07.787173 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:07.787146 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2"] Jul 10 22:51:07.789391 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:51:07.789363 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32f1d5d_f6fa_410f_ab26_b60faede8696.slice/crio-c3f3b41d3422a7530329317c11a88f92fde10d9693cb12e105111b9ddd2f7c01 WatchSource:0}: Error finding container c3f3b41d3422a7530329317c11a88f92fde10d9693cb12e105111b9ddd2f7c01: Status 404 returned error can't find the container with id c3f3b41d3422a7530329317c11a88f92fde10d9693cb12e105111b9ddd2f7c01 Jul 10 22:51:08.646825 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:08.646768 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" event={"ID":"e32f1d5d-f6fa-410f-ab26-b60faede8696","Type":"ContainerStarted","Data":"c3f3b41d3422a7530329317c11a88f92fde10d9693cb12e105111b9ddd2f7c01"} Jul 10 22:51:12.669875 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:12.669838 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" event={"ID":"e32f1d5d-f6fa-410f-ab26-b60faede8696","Type":"ContainerStarted","Data":"452ef5338229b20550162e54c5aeb7651d62a6f6080a454ad3c2d24010c71947"} Jul 10 22:51:12.670305 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:12.669920 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:12.686306 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:12.686263 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" podStartSLOduration=0.960267568 podStartE2EDuration="5.686249531s" totalImagesPullingTime="4.725981963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:51:12.684736215 +0000 UTC m=+531.322161890" watchObservedRunningTime="2026-07-10 22:51:12.686249531 +0000 UTC m=+531.323675233" Jul 10 22:51:18.703821 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:18.703795 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6sbt2" Jul 10 22:51:19.013675 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.013641 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:19.421175 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.421100 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:19.489686 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.489660 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bb796d979-g4zxs"] Jul 10 22:51:19.495523 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.495503 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:19.498423 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.498402 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m5d7b\"" Jul 10 22:51:19.499808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.499787 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bb796d979-g4zxs"] Jul 10 22:51:19.535921 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.535899 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lrrm\" (UniqueName: \"kubernetes.io/projected/4cc58c8b-a6ab-4850-8404-80a621157907-kube-api-access-2lrrm\") pod \"authorino-bb796d979-g4zxs\" (UID: \"4cc58c8b-a6ab-4850-8404-80a621157907\") " pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:19.637270 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.637242 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lrrm\" (UniqueName: \"kubernetes.io/projected/4cc58c8b-a6ab-4850-8404-80a621157907-kube-api-access-2lrrm\") pod \"authorino-bb796d979-g4zxs\" (UID: \"4cc58c8b-a6ab-4850-8404-80a621157907\") " pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:19.650502 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.650481 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lrrm\" (UniqueName: \"kubernetes.io/projected/4cc58c8b-a6ab-4850-8404-80a621157907-kube-api-access-2lrrm\") pod \"authorino-bb796d979-g4zxs\" (UID: \"4cc58c8b-a6ab-4850-8404-80a621157907\") " pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:19.794137 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.794105 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bb796d979-g4zxs"] Jul 10 22:51:19.794561 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.794369 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:19.926020 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:19.925997 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bb796d979-g4zxs"] Jul 10 22:51:19.928548 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:51:19.928521 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cc58c8b_a6ab_4850_8404_80a621157907.slice/crio-9a788012620baf5e9e6bafb5710d7c12cf050415073b9f32cd996aec97e7fc46 WatchSource:0}: Error finding container 9a788012620baf5e9e6bafb5710d7c12cf050415073b9f32cd996aec97e7fc46: Status 404 returned error can't find the container with id 9a788012620baf5e9e6bafb5710d7c12cf050415073b9f32cd996aec97e7fc46 Jul 10 22:51:20.718192 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:20.717975 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb796d979-g4zxs" event={"ID":"4cc58c8b-a6ab-4850-8404-80a621157907","Type":"ContainerStarted","Data":"9a788012620baf5e9e6bafb5710d7c12cf050415073b9f32cd996aec97e7fc46"} Jul 10 22:51:23.735833 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:23.735797 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb796d979-g4zxs" event={"ID":"4cc58c8b-a6ab-4850-8404-80a621157907","Type":"ContainerStarted","Data":"60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f"} Jul 10 22:51:23.736264 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:23.735846 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-bb796d979-g4zxs" podUID="4cc58c8b-a6ab-4850-8404-80a621157907" containerName="authorino" containerID="cri-o://60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f" gracePeriod=30 Jul 10 22:51:23.752537 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:23.752486 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-bb796d979-g4zxs" podStartSLOduration=1.778597395 podStartE2EDuration="4.752467118s" totalImagesPullingTime="2.973869723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:51:23.749386433 +0000 UTC m=+542.386812109" watchObservedRunningTime="2026-07-10 22:51:23.752467118 +0000 UTC m=+542.389892795" Jul 10 22:51:24.005862 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.005840 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:24.073779 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.073750 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lrrm\" (UniqueName: \"kubernetes.io/projected/4cc58c8b-a6ab-4850-8404-80a621157907-kube-api-access-2lrrm\") pod \"4cc58c8b-a6ab-4850-8404-80a621157907\" (UID: \"4cc58c8b-a6ab-4850-8404-80a621157907\") " Jul 10 22:51:24.075690 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.075669 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cc58c8b-a6ab-4850-8404-80a621157907-kube-api-access-2lrrm" (OuterVolumeSpecName: "kube-api-access-2lrrm") pod "4cc58c8b-a6ab-4850-8404-80a621157907" (UID: "4cc58c8b-a6ab-4850-8404-80a621157907"). InnerVolumeSpecName "kube-api-access-2lrrm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 22:51:24.174834 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.174809 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lrrm\" (UniqueName: \"kubernetes.io/projected/4cc58c8b-a6ab-4850-8404-80a621157907-kube-api-access-2lrrm\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 22:51:24.741649 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.741613 2551 generic.go:356] "Generic (PLEG): container finished" podID="4cc58c8b-a6ab-4850-8404-80a621157907" containerID="60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f" exitCode=0 Jul 10 22:51:24.742082 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.741652 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb796d979-g4zxs" event={"ID":"4cc58c8b-a6ab-4850-8404-80a621157907","Type":"ContainerDied","Data":"60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f"} Jul 10 22:51:24.742082 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.741696 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bb796d979-g4zxs" event={"ID":"4cc58c8b-a6ab-4850-8404-80a621157907","Type":"ContainerDied","Data":"9a788012620baf5e9e6bafb5710d7c12cf050415073b9f32cd996aec97e7fc46"} Jul 10 22:51:24.742082 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.741713 2551 scope.go:117] "RemoveContainer" containerID="60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f" Jul 10 22:51:24.742082 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.741655 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bb796d979-g4zxs" Jul 10 22:51:24.750737 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.750722 2551 scope.go:117] "RemoveContainer" containerID="60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f" Jul 10 22:51:24.750987 ip-10-0-138-188 kubenswrapper[2551]: E0710 22:51:24.750967 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f\": container with ID starting with 60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f not found: ID does not exist" containerID="60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f" Jul 10 22:51:24.751052 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.750998 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f"} err="failed to get container status \"60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f\": rpc error: code = NotFound desc = could not find container \"60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f\": container with ID starting with 60a9991b83f0a267e71cbee6c2cfbd717f01afd7120312384ac8ccca1e1cd39f not found: ID does not exist" Jul 10 22:51:24.764065 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.764033 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bb796d979-g4zxs"] Jul 10 22:51:24.765722 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:24.765699 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bb796d979-g4zxs"] Jul 10 22:51:25.246384 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:25.246348 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:25.408997 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:25.408961 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:25.510935 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:25.510897 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:25.608622 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:25.608580 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:26.041690 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:26.041663 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc58c8b-a6ab-4850-8404-80a621157907" path="/var/lib/kubelet/pods/4cc58c8b-a6ab-4850-8404-80a621157907/volumes" Jul 10 22:51:27.962131 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:27.962095 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:51:28.087221 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:51:28.087182 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:00.466753 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:00.466718 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:01.028279 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.028241 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:01.531559 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.531529 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:01.631588 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.631542 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:01.681808 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.681774 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-k4msn"] Jul 10 22:52:01.682229 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.682197 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc58c8b-a6ab-4850-8404-80a621157907" containerName="authorino" Jul 10 22:52:01.682324 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.682230 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc58c8b-a6ab-4850-8404-80a621157907" containerName="authorino" Jul 10 22:52:01.682382 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.682323 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc58c8b-a6ab-4850-8404-80a621157907" containerName="authorino" Jul 10 22:52:01.686390 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.686370 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.688915 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.688895 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r6zg4\"" Jul 10 22:52:01.689019 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.688895 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 22:52:01.695633 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.695605 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-k4msn"] Jul 10 22:52:01.699725 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.699646 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:01.718764 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.718741 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-d7zpj"] Jul 10 22:52:01.721166 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.721150 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.732781 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.732756 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-d7zpj"] Jul 10 22:52:01.772832 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.772801 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcljh\" (UniqueName: \"kubernetes.io/projected/53349d7d-81ab-46cf-88b3-75b931a737bd-kube-api-access-mcljh\") pod \"payload-pre-processing-74584bb9f8-k4msn\" (UID: \"53349d7d-81ab-46cf-88b3-75b931a737bd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.772959 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.772853 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53349d7d-81ab-46cf-88b3-75b931a737bd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-k4msn\" (UID: \"53349d7d-81ab-46cf-88b3-75b931a737bd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.788345 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.788287 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:52:01.873959 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.873912 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcljh\" (UniqueName: \"kubernetes.io/projected/53349d7d-81ab-46cf-88b3-75b931a737bd-kube-api-access-mcljh\") pod \"payload-pre-processing-74584bb9f8-k4msn\" (UID: \"53349d7d-81ab-46cf-88b3-75b931a737bd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.874155 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.874004 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53349d7d-81ab-46cf-88b3-75b931a737bd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-k4msn\" (UID: \"53349d7d-81ab-46cf-88b3-75b931a737bd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.874155 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.874039 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-d7zpj\" (UID: \"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.874385 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.874351 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k96vs\" (UniqueName: \"kubernetes.io/projected/a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1-kube-api-access-k96vs\") pod \"payload-processing-f9b8d9b9b-d7zpj\" (UID: \"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.875146 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.875119 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53349d7d-81ab-46cf-88b3-75b931a737bd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-k4msn\" (UID: \"53349d7d-81ab-46cf-88b3-75b931a737bd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.882606 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.882585 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcljh\" (UniqueName: \"kubernetes.io/projected/53349d7d-81ab-46cf-88b3-75b931a737bd-kube-api-access-mcljh\") pod \"payload-pre-processing-74584bb9f8-k4msn\" (UID: \"53349d7d-81ab-46cf-88b3-75b931a737bd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:01.976438 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.976409 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-d7zpj\" (UID: \"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.976438 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.976442 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k96vs\" (UniqueName: \"kubernetes.io/projected/a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1-kube-api-access-k96vs\") pod \"payload-processing-f9b8d9b9b-d7zpj\" (UID: \"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.977098 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.977076 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-d7zpj\" (UID: \"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.984286 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.984252 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k96vs\" (UniqueName: \"kubernetes.io/projected/a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1-kube-api-access-k96vs\") pod \"payload-processing-f9b8d9b9b-d7zpj\" (UID: \"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:01.997048 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:01.997029 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:02.031340 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:02.031308 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:02.156122 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:02.154829 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-k4msn"] Jul 10 22:52:02.165310 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:52:02.165282 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53349d7d_81ab_46cf_88b3_75b931a737bd.slice/crio-79487b7f71cc0adea36f687505f7b86f1e6c2717ee1702cba2a8f7a4d29a0bca WatchSource:0}: Error finding container 79487b7f71cc0adea36f687505f7b86f1e6c2717ee1702cba2a8f7a4d29a0bca: Status 404 returned error can't find the container with id 79487b7f71cc0adea36f687505f7b86f1e6c2717ee1702cba2a8f7a4d29a0bca Jul 10 22:52:02.198125 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:02.198102 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-d7zpj"] Jul 10 22:52:02.199663 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:52:02.199632 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ab5dcf_953b_48eb_b6e5_ea03fda6fad1.slice/crio-bce659fdbc55519d6f75fd76157ea043a874f460336cd854be283d257f280b36 WatchSource:0}: Error finding container bce659fdbc55519d6f75fd76157ea043a874f460336cd854be283d257f280b36: Status 404 returned error can't find the container with id bce659fdbc55519d6f75fd76157ea043a874f460336cd854be283d257f280b36 Jul 10 22:52:02.916157 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:02.916116 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" event={"ID":"53349d7d-81ab-46cf-88b3-75b931a737bd","Type":"ContainerStarted","Data":"79487b7f71cc0adea36f687505f7b86f1e6c2717ee1702cba2a8f7a4d29a0bca"} Jul 10 22:52:02.917767 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:02.917738 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" event={"ID":"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1","Type":"ContainerStarted","Data":"bce659fdbc55519d6f75fd76157ea043a874f460336cd854be283d257f280b36"} Jul 10 22:52:03.925444 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:03.925349 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" event={"ID":"a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1","Type":"ContainerStarted","Data":"fd528c31c205f37ce115e6ac6be4bf7b772dd095de827be3492be45fbeab4591"} Jul 10 22:52:03.925444 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:03.925424 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:03.927028 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:03.927007 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" event={"ID":"53349d7d-81ab-46cf-88b3-75b931a737bd","Type":"ContainerStarted","Data":"1a73893939eeb48a2d85d3daeb92dc37cc35dc4b4757600b298e5f23e2eff1d1"} Jul 10 22:52:03.927136 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:03.927091 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:03.940515 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:03.940459 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" podStartSLOduration=1.45072599 podStartE2EDuration="2.940445314s" totalImagesPullingTime="1.489719324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:52:03.939275415 +0000 UTC m=+582.576701091" watchObservedRunningTime="2026-07-10 22:52:03.940445314 +0000 UTC m=+582.577870988" Jul 10 22:52:03.954749 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:03.954710 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" podStartSLOduration=1.427094963 podStartE2EDuration="2.954697173s" totalImagesPullingTime="1.52760221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:52:03.953163637 +0000 UTC m=+582.590589323" watchObservedRunningTime="2026-07-10 22:52:03.954697173 +0000 UTC m=+582.592122856" Jul 10 22:52:09.958341 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:09.958306 2551 generic.go:356] "Generic (PLEG): container finished" podID="27ab5b4c-6e90-4d13-99f4-bff938e21213" containerID="e5779d17a0dc6bccea2105e701dc8c61d0f28ba2011818972d87a62d2e92c108" exitCode=0 Jul 10 22:52:09.958748 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:09.958380 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" event={"ID":"27ab5b4c-6e90-4d13-99f4-bff938e21213","Type":"ContainerDied","Data":"e5779d17a0dc6bccea2105e701dc8c61d0f28ba2011818972d87a62d2e92c108"} Jul 10 22:52:09.958838 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:09.958749 2551 scope.go:117] "RemoveContainer" containerID="e5779d17a0dc6bccea2105e701dc8c61d0f28ba2011818972d87a62d2e92c108" Jul 10 22:52:09.959725 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:09.959698 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:52:09.959779 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:09.959762 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:52:09.959819 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:09.959796 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 22:52:10.965056 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:10.965024 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" event={"ID":"27ab5b4c-6e90-4d13-99f4-bff938e21213","Type":"ContainerStarted","Data":"b7a7fc7c9bcb8d678abb2683d8b216496fe4186ff32a3dd5279540663730a4b5"} Jul 10 22:52:11.034083 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:11.034050 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:52:11.034408 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:11.034385 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lpbbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 22:52:11.034464 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:11.034443 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" podUID="27ab5b4c-6e90-4d13-99f4-bff938e21213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 22:52:11.239661 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:11.239583 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:11.332851 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:11.332817 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:12.033562 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:12.033517 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lpbbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 22:52:12.034028 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:12.033579 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" podUID="27ab5b4c-6e90-4d13-99f4-bff938e21213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 22:52:13.033835 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:13.033799 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lpbbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 22:52:13.034238 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:13.033852 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" podUID="27ab5b4c-6e90-4d13-99f4-bff938e21213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 22:52:14.033767 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:14.033730 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lpbbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 10 22:52:14.033951 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:14.033784 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" podUID="27ab5b4c-6e90-4d13-99f4-bff938e21213" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 10 22:52:14.933960 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:14.933926 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7zpj" Jul 10 22:52:14.934149 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:14.934012 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-k4msn" Jul 10 22:52:15.036965 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:15.036934 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:52:15.037359 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:15.037306 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:52:15.037918 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:15.037902 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lpbbh" Jul 10 22:52:18.061975 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:18.061933 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:18.539348 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:18.539319 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:52:19.992386 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:19.992352 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:21.960490 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:21.960461 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:52:21.967840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:21.967816 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:52:25.857089 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:25.857045 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:26.695747 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:26.695716 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:52:35.965147 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:35.965108 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:37.094054 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:37.094005 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:52:43.630269 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.630236 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45"] Jul 10 22:52:43.636663 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.636636 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.639501 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.639475 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 22:52:43.639842 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.639567 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 22:52:43.640705 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.640679 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-crclv\"" Jul 10 22:52:43.640839 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.640729 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 22:52:43.644434 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.644406 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45"] Jul 10 22:52:43.662999 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.662960 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:43.754978 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.754906 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.754978 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.754951 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbg2c\" (UniqueName: \"kubernetes.io/projected/2824666d-59d9-40d9-a3c1-a3d0b2e01491-kube-api-access-zbg2c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.755152 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.755005 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.755152 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.755022 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2824666d-59d9-40d9-a3c1-a3d0b2e01491-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.755152 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.755122 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.755313 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.755179 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.855646 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.855607 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.855646 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.855652 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.855840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.855685 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.855840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.855702 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbg2c\" (UniqueName: \"kubernetes.io/projected/2824666d-59d9-40d9-a3c1-a3d0b2e01491-kube-api-access-zbg2c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.855840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.855728 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.855840 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.855743 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2824666d-59d9-40d9-a3c1-a3d0b2e01491-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.856116 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.856088 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.856266 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.856134 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.856266 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.856180 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.857895 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.857870 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2824666d-59d9-40d9-a3c1-a3d0b2e01491-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.858169 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.858150 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2824666d-59d9-40d9-a3c1-a3d0b2e01491-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.863865 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.863847 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbg2c\" (UniqueName: \"kubernetes.io/projected/2824666d-59d9-40d9-a3c1-a3d0b2e01491-kube-api-access-zbg2c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45\" (UID: \"2824666d-59d9-40d9-a3c1-a3d0b2e01491\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:43.948935 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:43.948876 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:44.080724 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:44.080702 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45"] Jul 10 22:52:44.082788 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:52:44.082759 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2824666d_59d9_40d9_a3c1_a3d0b2e01491.slice/crio-98299ed176cfb6b1d86f974b90861ae90bd70f9b5718a8f86e1f7c2786de872d WatchSource:0}: Error finding container 98299ed176cfb6b1d86f974b90861ae90bd70f9b5718a8f86e1f7c2786de872d: Status 404 returned error can't find the container with id 98299ed176cfb6b1d86f974b90861ae90bd70f9b5718a8f86e1f7c2786de872d Jul 10 22:52:44.084091 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:44.084028 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 22:52:44.134763 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:44.134734 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" event={"ID":"2824666d-59d9-40d9-a3c1-a3d0b2e01491","Type":"ContainerStarted","Data":"98299ed176cfb6b1d86f974b90861ae90bd70f9b5718a8f86e1f7c2786de872d"} Jul 10 22:52:45.107007 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:45.106977 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:52:46.145862 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:46.145824 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" event={"ID":"2824666d-59d9-40d9-a3c1-a3d0b2e01491","Type":"ContainerStarted","Data":"5aaa93a1575dc6730767e32ca0722cabe2284f7bc24d9d379dc327b61343a4cd"} Jul 10 22:52:53.949199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:53.949161 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:53.949199 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:53.949231 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:53.961283 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:53.961258 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:53.978768 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:53.978724 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" podStartSLOduration=9.984262543 podStartE2EDuration="10.978711527s" totalImagesPullingTime="994.448984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:52:46.18107554 +0000 UTC m=+624.818501215" watchObservedRunningTime="2026-07-10 22:52:53.978711527 +0000 UTC m=+632.616137192" Jul 10 22:52:54.194868 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:54.194847 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45" Jul 10 22:52:55.878113 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:55.878073 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:52:57.897177 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:52:57.897144 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:53:07.822996 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:53:07.822961 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:53:10.289226 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:53:10.289189 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:53:32.483349 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:53:32.483320 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:53:43.849510 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:53:43.849465 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:53:43.902725 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:53:43.902679 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:53:53.341687 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:53:53.341648 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:04.292526 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.292486 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55b7458fdb-btdhh"] Jul 10 22:54:04.296129 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.296104 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.299381 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.299356 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 22:54:04.299381 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.299378 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m5d7b\"" Jul 10 22:54:04.304020 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.303601 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55b7458fdb-btdhh"] Jul 10 22:54:04.340024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.339974 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/27167415-6a5d-4c1b-a196-c74e9ab0a0ad-tls-cert\") pod \"authorino-55b7458fdb-btdhh\" (UID: \"27167415-6a5d-4c1b-a196-c74e9ab0a0ad\") " pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.340024 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.340027 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf9pt\" (UniqueName: \"kubernetes.io/projected/27167415-6a5d-4c1b-a196-c74e9ab0a0ad-kube-api-access-hf9pt\") pod \"authorino-55b7458fdb-btdhh\" (UID: \"27167415-6a5d-4c1b-a196-c74e9ab0a0ad\") " pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.440681 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.440641 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/27167415-6a5d-4c1b-a196-c74e9ab0a0ad-tls-cert\") pod \"authorino-55b7458fdb-btdhh\" (UID: \"27167415-6a5d-4c1b-a196-c74e9ab0a0ad\") " pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.440681 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.440682 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf9pt\" (UniqueName: \"kubernetes.io/projected/27167415-6a5d-4c1b-a196-c74e9ab0a0ad-kube-api-access-hf9pt\") pod \"authorino-55b7458fdb-btdhh\" (UID: \"27167415-6a5d-4c1b-a196-c74e9ab0a0ad\") " pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.443436 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.443406 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/27167415-6a5d-4c1b-a196-c74e9ab0a0ad-tls-cert\") pod \"authorino-55b7458fdb-btdhh\" (UID: \"27167415-6a5d-4c1b-a196-c74e9ab0a0ad\") " pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.449849 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.449821 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf9pt\" (UniqueName: \"kubernetes.io/projected/27167415-6a5d-4c1b-a196-c74e9ab0a0ad-kube-api-access-hf9pt\") pod \"authorino-55b7458fdb-btdhh\" (UID: \"27167415-6a5d-4c1b-a196-c74e9ab0a0ad\") " pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.607283 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.607149 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b7458fdb-btdhh" Jul 10 22:54:04.705095 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.705040 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:04.767801 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.767773 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55b7458fdb-btdhh"] Jul 10 22:54:04.771996 ip-10-0-138-188 kubenswrapper[2551]: W0710 22:54:04.771962 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27167415_6a5d_4c1b_a196_c74e9ab0a0ad.slice/crio-85ce4658bfdc701f9f9f3d3027a0c7a4bb59f84d14bae6234fa6afd956ddca89 WatchSource:0}: Error finding container 85ce4658bfdc701f9f9f3d3027a0c7a4bb59f84d14bae6234fa6afd956ddca89: Status 404 returned error can't find the container with id 85ce4658bfdc701f9f9f3d3027a0c7a4bb59f84d14bae6234fa6afd956ddca89 Jul 10 22:54:04.801342 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:04.801309 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:05.535807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:05.535764 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b7458fdb-btdhh" event={"ID":"27167415-6a5d-4c1b-a196-c74e9ab0a0ad","Type":"ContainerStarted","Data":"d647c1c966e1636a176d6b37ecc8e0dfd9bd3d18c72098b1eecbc14d52fd13c4"} Jul 10 22:54:05.535807 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:05.535808 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b7458fdb-btdhh" event={"ID":"27167415-6a5d-4c1b-a196-c74e9ab0a0ad","Type":"ContainerStarted","Data":"85ce4658bfdc701f9f9f3d3027a0c7a4bb59f84d14bae6234fa6afd956ddca89"} Jul 10 22:54:05.568734 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:05.568684 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55b7458fdb-btdhh" podStartSLOduration=1.115043789 podStartE2EDuration="1.568665056s" totalImagesPullingTime="453.621267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:54:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:54:05.560395422 +0000 UTC m=+704.197821098" watchObservedRunningTime="2026-07-10 22:54:05.568665056 +0000 UTC m=+704.206090731" Jul 10 22:54:05.894091 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:05.894005 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:05.977811 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:05.977772 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:06.963009 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:06.962954 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:29.014444 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:29.014404 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:29.122295 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:29.122243 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:54:32.196405 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:32.196362 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:54:43.007058 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:43.007012 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:54:53.395472 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:53.395426 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:54:54.020335 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:54:54.020287 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:03.189621 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:03.189569 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:55:13.700105 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:13.700063 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:55:15.718375 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:15.718338 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:16.966536 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:16.966498 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:50.717601 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:50.717503 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:50.822768 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:50.822729 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:50.930429 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:50.930392 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:58.496843 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:58.496804 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:58.806394 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:58.806349 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:55:58.949682 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:55:58.949647 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:56:04.768318 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:04.768278 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:56:04.850936 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:04.850893 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:56:20.295595 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:20.295550 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:56:29.791643 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:29.791603 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:56:37.598582 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:37.598537 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:56:45.397978 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:45.397936 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:56:54.699589 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:56:54.699542 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:57:03.897252 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:03.897202 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:57:16.014753 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:16.014713 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:57:21.994006 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:21.993974 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:57:21.997663 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:21.997639 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 22:57:26.400329 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:26.400259 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:57:47.962974 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:47.962899 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:57:56.891994 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:57:56.891953 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:58:20.597195 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:58:20.597157 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:58:32.097267 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:58:32.097228 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:58:47.920269 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:58:47.920161 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:58:48.028479 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:58:48.028438 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 22:58:57.295410 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:58:57.295367 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:59:06.799875 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:59:06.799837 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:59:15.993354 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:59:15.993316 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:59:25.602598 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:59:25.602551 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 22:59:43.114694 ip-10-0-138-188 kubenswrapper[2551]: I0710 22:59:43.114653 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:01:12.197430 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:01:12.197387 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:01:23.798565 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:01:23.798527 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:01:33.190104 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:01:33.190033 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:01:44.102265 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:01:44.102141 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:01:56.391360 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:01:56.391318 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:05.596354 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:05.596303 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:15.910905 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:15.910859 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:22.035037 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:22.035001 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:02:22.035514 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:22.035000 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:02:25.538889 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:25.538845 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:36.296792 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:36.296748 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:38.988956 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:38.988912 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:02:44.495935 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:44.495897 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:53.708594 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:53.708535 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:02:59.894852 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:02:59.894808 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:03:25.202096 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:03:25.202054 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:03:32.606009 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:03:32.605967 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:03:42.303173 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:03:42.303135 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:03:51.790903 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:03:51.790857 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:03:52.775094 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:03:52.775055 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:04:09.195971 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:04:09.195934 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:04:20.599358 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:04:20.599317 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:04:30.193517 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:04:30.193473 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:04:37.995553 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:04:37.995516 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:04:48.400413 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:04:48.400294 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:04:57.397650 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:04:57.397608 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:05:10.898538 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:05:10.898496 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:05:19.905900 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:05:19.905856 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:05:29.794035 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:05:29.793998 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:05:41.992097 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:05:41.992059 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:05:51.189882 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:05:51.189841 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:06:01.697849 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:01.697802 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:06:13.591624 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:13.591584 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:06:24.405724 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:24.405682 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:06:38.104664 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:38.104602 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:06:40.271430 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:40.271382 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:06:47.657499 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:47.657463 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:06:51.601303 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:51.601256 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:06:56.633410 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:06:56.633365 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:00.993809 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:00.993764 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:07:01.047535 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:01.047496 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:03.942583 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:03.942534 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:04.037062 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:04.037022 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:04.347169 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:04.347124 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:10.163760 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.163721 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:10.271277 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.271235 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:10.429195 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.429103 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w"] Jul 10 23:07:10.432600 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.432573 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.435947 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.435921 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8361fb0e-openshift-default-dockercfg-lbc64\"" Jul 10 23:07:10.463587 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.463477 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:10.465673 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.465643 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w"] Jul 10 23:07:10.468121 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.468085 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:07:10.576485 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.576449 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:10.587701 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587664 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-socket\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587735 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-data\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587758 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-certs\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587781 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9d5a91-6a94-4570-8868-5111537b8168-istiod-ca-cert\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587797 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-envoy\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587813 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-istio-token\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587988 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587876 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-credential-socket\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587988 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587904 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m66f2\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-kube-api-access-m66f2\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.587988 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.587972 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9d5a91-6a94-4570-8868-5111537b8168-istio-podinfo\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689259 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689140 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-data\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689259 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689184 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-certs\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689259 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689226 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9d5a91-6a94-4570-8868-5111537b8168-istiod-ca-cert\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689259 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689250 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-envoy\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689274 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-istio-token\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689313 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-credential-socket\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689354 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m66f2\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-kube-api-access-m66f2\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689439 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9d5a91-6a94-4570-8868-5111537b8168-istio-podinfo\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689526 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-socket\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689835 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689620 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-data\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689835 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689678 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-certs\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689957 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9d5a91-6a94-4570-8868-5111537b8168-istiod-ca-cert\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.689980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.689968 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-credential-socket\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.690093 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.690037 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-socket\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.691863 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.691845 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-envoy\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.692132 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.692113 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9d5a91-6a94-4570-8868-5111537b8168-istio-podinfo\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.698768 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.698741 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-istio-token\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.699057 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.699030 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m66f2\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-kube-api-access-m66f2\") pod \"e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.743719 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.743679 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:10.894886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.894857 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w"] Jul 10 23:07:10.896467 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:07:10.896440 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c9d5a91_6a94_4570_8868_5111537b8168.slice/crio-29e40c9d059ba84254c79184d517fd657dcb293ed0b350eefcb38d7f09bbb00d WatchSource:0}: Error finding container 29e40c9d059ba84254c79184d517fd657dcb293ed0b350eefcb38d7f09bbb00d: Status 404 returned error can't find the container with id 29e40c9d059ba84254c79184d517fd657dcb293ed0b350eefcb38d7f09bbb00d Jul 10 23:07:10.898940 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.898907 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:10.899031 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.898979 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:10.899031 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:10.899016 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:11.594980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.594942 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:11.628918 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.628857 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" event={"ID":"3c9d5a91-6a94-4570-8868-5111537b8168","Type":"ContainerStarted","Data":"851a5589aa3a6bb0ce2aa499924a5f6e256738eaa551b5f96e8b11de8397a8c0"} Jul 10 23:07:11.629118 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.628974 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" event={"ID":"3c9d5a91-6a94-4570-8868-5111537b8168","Type":"ContainerStarted","Data":"29e40c9d059ba84254c79184d517fd657dcb293ed0b350eefcb38d7f09bbb00d"} Jul 10 23:07:11.653025 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.652962 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" podStartSLOduration=1.652939749 podStartE2EDuration="1.652939749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:07:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:07:11.651681221 +0000 UTC m=+1490.289106909" watchObservedRunningTime="2026-07-10 23:07:11.652939749 +0000 UTC m=+1490.290365426" Jul 10 23:07:11.704330 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.704288 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:11.744417 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.744379 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:11.749691 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.749655 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:11.782814 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:11.782777 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:12.110701 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:12.110661 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:12.257583 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:12.257533 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:12.358627 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:12.358586 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:12.634349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:12.634320 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:12.635385 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:12.635364 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:14.205017 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:14.204983 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:16.609971 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:16.609935 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:16.803490 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:16.803452 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w"] Jul 10 23:07:16.803685 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:16.803658 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" podUID="3c9d5a91-6a94-4570-8868-5111537b8168" containerName="istio-proxy" containerID="cri-o://851a5589aa3a6bb0ce2aa499924a5f6e256738eaa551b5f96e8b11de8397a8c0" gracePeriod=30 Jul 10 23:07:16.832842 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:16.832802 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:16.935304 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:16.935173 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:17.192985 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:17.192878 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:18.684953 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:18.684917 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:22.075526 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.075494 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:07:22.076028 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.075932 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:07:22.735600 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.735561 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:22.836080 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.836045 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:22.894816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.894778 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6"] Jul 10 23:07:22.899000 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.898968 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:22.902125 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.902101 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-86f71747-openshift-default-dockercfg-kdwpk\"" Jul 10 23:07:22.915940 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.915900 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6"] Jul 10 23:07:22.943745 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:22.943692 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:23.002042 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002007 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-token\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002238 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002088 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-socket\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002238 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002121 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-podinfo\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002238 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002172 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52dr9\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-kube-api-access-52dr9\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002238 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002203 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-credential-socket\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002400 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-envoy\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002481 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002404 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istiod-ca-cert\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002481 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002466 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-certs\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.002556 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.002496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-data\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.043020 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.042983 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:23.103967 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.103925 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-socket\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.103967 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.103972 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-podinfo\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104575 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104016 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dr9\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-kube-api-access-52dr9\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104575 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104046 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-credential-socket\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104575 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104085 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-envoy\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104575 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104450 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-socket\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104575 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104472 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-credential-socket\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104804 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104672 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istiod-ca-cert\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104804 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104764 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-certs\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104804 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104796 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-data\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.104970 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.104846 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-token\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.105130 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.105100 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-certs\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.105264 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.105142 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-data\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.105697 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.105673 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istiod-ca-cert\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.106779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.106759 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-envoy\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.107362 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.107338 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-podinfo\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.113080 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.113036 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52dr9\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-kube-api-access-52dr9\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.113261 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.113087 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-token\") pod \"e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.212158 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.212116 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:23.379436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.379307 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6"] Jul 10 23:07:23.382387 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:07:23.382356 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e3ff7a_4de8_4ce7_a009_49a2c2f23a67.slice/crio-bb9449f7d2882c73963f03ba964e8931e1a2952b7ce3918a23f750b0de713a80 WatchSource:0}: Error finding container bb9449f7d2882c73963f03ba964e8931e1a2952b7ce3918a23f750b0de713a80: Status 404 returned error can't find the container with id bb9449f7d2882c73963f03ba964e8931e1a2952b7ce3918a23f750b0de713a80 Jul 10 23:07:23.384939 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.384897 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:23.385039 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.384989 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:23.385086 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.385038 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:23.717365 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.717314 2551 generic.go:356] "Generic (PLEG): container finished" podID="3c9d5a91-6a94-4570-8868-5111537b8168" containerID="851a5589aa3a6bb0ce2aa499924a5f6e256738eaa551b5f96e8b11de8397a8c0" exitCode=0 Jul 10 23:07:23.717609 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.717467 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" event={"ID":"3c9d5a91-6a94-4570-8868-5111537b8168","Type":"ContainerDied","Data":"851a5589aa3a6bb0ce2aa499924a5f6e256738eaa551b5f96e8b11de8397a8c0"} Jul 10 23:07:23.720064 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.719931 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" event={"ID":"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67","Type":"ContainerStarted","Data":"ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885"} Jul 10 23:07:23.720064 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.720008 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" event={"ID":"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67","Type":"ContainerStarted","Data":"bb9449f7d2882c73963f03ba964e8931e1a2952b7ce3918a23f750b0de713a80"} Jul 10 23:07:23.746507 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.746450 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" podStartSLOduration=1.7464243050000001 podStartE2EDuration="1.746424305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:07:23.745314664 +0000 UTC m=+1502.382740350" watchObservedRunningTime="2026-07-10 23:07:23.746424305 +0000 UTC m=+1502.383849981" Jul 10 23:07:23.753818 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.753788 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:23.813371 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813334 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-socket\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813371 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813374 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-credential-socket\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813428 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9d5a91-6a94-4570-8868-5111537b8168-istiod-ca-cert\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813449 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-istio-token\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813491 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9d5a91-6a94-4570-8868-5111537b8168-istio-podinfo\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813603 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-data\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813629 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m66f2\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-kube-api-access-m66f2\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813902 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813653 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-envoy\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813902 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813690 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-certs\") pod \"3c9d5a91-6a94-4570-8868-5111537b8168\" (UID: \"3c9d5a91-6a94-4570-8868-5111537b8168\") " Jul 10 23:07:23.813902 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813712 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:23.814071 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.813908 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c9d5a91-6a94-4570-8868-5111537b8168-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:07:23.814271 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.814245 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.814369 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.814274 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9d5a91-6a94-4570-8868-5111537b8168-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.814571 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.814514 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:23.814653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.814569 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:23.814714 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.814674 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:23.816098 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.816073 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:07:23.816246 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.816158 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-kube-api-access-m66f2" (OuterVolumeSpecName: "kube-api-access-m66f2") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "kube-api-access-m66f2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:07:23.816343 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.816322 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:23.816524 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.816503 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c9d5a91-6a94-4570-8868-5111537b8168-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c9d5a91-6a94-4570-8868-5111537b8168" (UID: "3c9d5a91-6a94-4570-8868-5111537b8168"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914859 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914894 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9d5a91-6a94-4570-8868-5111537b8168-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914904 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914914 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m66f2\" (UniqueName: \"kubernetes.io/projected/3c9d5a91-6a94-4570-8868-5111537b8168-kube-api-access-m66f2\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914924 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914932 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:23.914963 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:23.914940 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9d5a91-6a94-4570-8868-5111537b8168-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:24.039028 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.038982 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:24.133235 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.133179 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:24.212536 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.212432 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:24.213327 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.213297 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:24.217808 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.217780 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:24.599176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.599141 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:24.701161 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.701114 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:24.726578 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.726534 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" Jul 10 23:07:24.726780 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.726545 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w" event={"ID":"3c9d5a91-6a94-4570-8868-5111537b8168","Type":"ContainerDied","Data":"29e40c9d059ba84254c79184d517fd657dcb293ed0b350eefcb38d7f09bbb00d"} Jul 10 23:07:24.726780 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.726712 2551 scope.go:117] "RemoveContainer" containerID="851a5589aa3a6bb0ce2aa499924a5f6e256738eaa551b5f96e8b11de8397a8c0" Jul 10 23:07:24.727175 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.727098 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:24.728497 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.728470 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:24.777154 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.777072 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w"] Jul 10 23:07:24.782463 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.782427 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8361fb0e-openshift-default-59c8c5d46f-4cq2w"] Jul 10 23:07:24.883810 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:24.883726 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:26.042667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:26.042631 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c9d5a91-6a94-4570-8868-5111537b8168" path="/var/lib/kubelet/pods/3c9d5a91-6a94-4570-8868-5111537b8168/volumes" Jul 10 23:07:27.198332 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:27.198294 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:07:28.254022 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:28.253987 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:29.191709 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:29.191668 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:29.531788 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:29.531750 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:35.257413 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.257367 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6"] Jul 10 23:07:35.257865 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.257598 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" podUID="83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" containerName="istio-proxy" containerID="cri-o://ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885" gracePeriod=30 Jul 10 23:07:35.288798 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.288711 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:35.387939 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.387900 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:35.405741 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.405712 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:35.538197 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538163 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-podinfo\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.538197 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538202 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istiod-ca-cert\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.538698 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538671 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:07:35.538790 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538766 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-envoy\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.538841 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538812 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-data\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.538902 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538848 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-credential-socket\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.538902 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538893 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-socket\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.539007 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538915 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-certs\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.539007 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538974 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-token\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.539007 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.538998 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52dr9\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-kube-api-access-52dr9\") pod \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\" (UID: \"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67\") " Jul 10 23:07:35.539180 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.539158 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:35.539473 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.539416 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:35.539473 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.539434 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.539656 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.539472 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.539656 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.539578 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:35.539790 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.539761 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-data" (OuterVolumeSpecName: "istio-data") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:35.541115 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.541061 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:07:35.541382 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.541353 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:35.541528 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.541463 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-token" (OuterVolumeSpecName: "istio-token") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:07:35.541629 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.541614 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-kube-api-access-52dr9" (OuterVolumeSpecName: "kube-api-access-52dr9") pod "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" (UID: "83e3ff7a-4de8-4ce7-a009-49a2c2f23a67"). InnerVolumeSpecName "kube-api-access-52dr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640336 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640371 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640381 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640389 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640398 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640408 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.640433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.640416 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52dr9\" (UniqueName: \"kubernetes.io/projected/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67-kube-api-access-52dr9\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:35.800338 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.800299 2551 generic.go:356] "Generic (PLEG): container finished" podID="83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" containerID="ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885" exitCode=0 Jul 10 23:07:35.800338 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.800339 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" event={"ID":"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67","Type":"ContainerDied","Data":"ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885"} Jul 10 23:07:35.800559 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.800349 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" Jul 10 23:07:35.800559 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.800387 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6" event={"ID":"83e3ff7a-4de8-4ce7-a009-49a2c2f23a67","Type":"ContainerDied","Data":"bb9449f7d2882c73963f03ba964e8931e1a2952b7ce3918a23f750b0de713a80"} Jul 10 23:07:35.800559 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.800409 2551 scope.go:117] "RemoveContainer" containerID="ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885" Jul 10 23:07:35.811192 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.811174 2551 scope.go:117] "RemoveContainer" containerID="ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885" Jul 10 23:07:35.811521 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:07:35.811501 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885\": container with ID starting with ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885 not found: ID does not exist" containerID="ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885" Jul 10 23:07:35.811583 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.811532 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885"} err="failed to get container status \"ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885\": rpc error: code = NotFound desc = could not find container \"ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885\": container with ID starting with ddfad5961db95a44aaeb0af5362dc1168d4c0a2caa0be502e83a131aa098b885 not found: ID does not exist" Jul 10 23:07:35.832155 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.832115 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6"] Jul 10 23:07:35.834730 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:35.834697 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-86f71747-openshift-default-8497c5ff7f-2c9l6"] Jul 10 23:07:36.042201 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:36.042163 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" path="/var/lib/kubelet/pods/83e3ff7a-4de8-4ce7-a009-49a2c2f23a67/volumes" Jul 10 23:07:38.125150 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:38.125107 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:07:39.504600 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:39.504566 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:41.123440 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.123403 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:41.227740 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.227711 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:41.296816 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.296774 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2"] Jul 10 23:07:41.297320 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.297300 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9d5a91-6a94-4570-8868-5111537b8168" containerName="istio-proxy" Jul 10 23:07:41.297436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.297321 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9d5a91-6a94-4570-8868-5111537b8168" containerName="istio-proxy" Jul 10 23:07:41.297436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.297368 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" containerName="istio-proxy" Jul 10 23:07:41.297436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.297377 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" containerName="istio-proxy" Jul 10 23:07:41.297592 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.297496 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="83e3ff7a-4de8-4ce7-a009-49a2c2f23a67" containerName="istio-proxy" Jul 10 23:07:41.297592 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.297512 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c9d5a91-6a94-4570-8868-5111537b8168" containerName="istio-proxy" Jul 10 23:07:41.300831 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.300800 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.303333 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.303305 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b379716f-openshift-default-dockercfg-frx2n\"" Jul 10 23:07:41.315125 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.315081 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2"] Jul 10 23:07:41.336367 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.336334 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:41.396319 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396190 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-envoy\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396319 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396268 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-data\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396350 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-socket\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396367 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-token\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396387 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-certs\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396409 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/644ff901-6b0b-4676-b3ce-280ca8e34950-istiod-ca-cert\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396460 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djpwl\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-kube-api-access-djpwl\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396499 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-podinfo\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.396716 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.396528 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-credential-socket\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.437359 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.437321 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:41.497912 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.497874 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-socket\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.497912 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.497914 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-token\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.497936 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-certs\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.497993 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/644ff901-6b0b-4676-b3ce-280ca8e34950-istiod-ca-cert\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498042 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djpwl\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-kube-api-access-djpwl\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498072 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-podinfo\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498110 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-credential-socket\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498493 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498193 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-envoy\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498493 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498261 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-data\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498493 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498399 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-socket\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498531 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-certs\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498716 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498657 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-credential-socket\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498767 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498728 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-data\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.498864 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.498843 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/644ff901-6b0b-4676-b3ce-280ca8e34950-istiod-ca-cert\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.500733 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.500707 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-envoy\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.500980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.500958 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-podinfo\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.506671 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.506633 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-token\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.506848 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.506822 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djpwl\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-kube-api-access-djpwl\") pod \"e2e-derive-b379716f-openshift-default-7776fd587-tztm2\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.612961 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.612915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:41.765086 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.765052 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2"] Jul 10 23:07:41.767067 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:07:41.767036 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644ff901_6b0b_4676_b3ce_280ca8e34950.slice/crio-1ad440def21d15118a5c0bfb5a91a8fb75e4ba197b83388c6440407058a8f220 WatchSource:0}: Error finding container 1ad440def21d15118a5c0bfb5a91a8fb75e4ba197b83388c6440407058a8f220: Status 404 returned error can't find the container with id 1ad440def21d15118a5c0bfb5a91a8fb75e4ba197b83388c6440407058a8f220 Jul 10 23:07:41.769604 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.769559 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:41.769693 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.769652 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:41.769744 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.769691 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:07:41.835018 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.834978 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" event={"ID":"644ff901-6b0b-4676-b3ce-280ca8e34950","Type":"ContainerStarted","Data":"faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1"} Jul 10 23:07:41.835197 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.835042 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" event={"ID":"644ff901-6b0b-4676-b3ce-280ca8e34950","Type":"ContainerStarted","Data":"1ad440def21d15118a5c0bfb5a91a8fb75e4ba197b83388c6440407058a8f220"} Jul 10 23:07:41.862326 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:41.862096 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" podStartSLOduration=0.862069258 podStartE2EDuration="862.069258ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:07:41.853281542 +0000 UTC m=+1520.490707217" watchObservedRunningTime="2026-07-10 23:07:41.862069258 +0000 UTC m=+1520.499494933" Jul 10 23:07:42.446098 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.446056 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:42.613153 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.613116 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:42.614092 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.614057 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:42.619121 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.619089 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:42.790083 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.790047 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:42.840557 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.840529 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:42.841725 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:42.841700 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:43.064559 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:43.064436 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:43.344895 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:43.344793 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:43.444862 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:43.444826 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:44.612269 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:44.612234 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:46.869455 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:46.869417 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:47.059307 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:47.059271 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2"] Jul 10 23:07:47.059555 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:47.059527 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" podUID="644ff901-6b0b-4676-b3ce-280ca8e34950" containerName="istio-proxy" containerID="cri-o://faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1" gracePeriod=30 Jul 10 23:07:47.086601 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:47.086564 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:47.184446 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:47.184351 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:47.484848 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:47.484760 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:48.755587 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:48.755552 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:52.331174 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.331088 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:52.404311 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404268 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-socket\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404525 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404319 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-data\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404525 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404352 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-credential-socket\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404525 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404387 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djpwl\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-kube-api-access-djpwl\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404525 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404477 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-envoy\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404525 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404509 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-token\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404592 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-podinfo\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404613 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:52.404805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404703 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/644ff901-6b0b-4676-b3ce-280ca8e34950-istiod-ca-cert\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404735 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-certs\") pod \"644ff901-6b0b-4676-b3ce-280ca8e34950\" (UID: \"644ff901-6b0b-4676-b3ce-280ca8e34950\") " Jul 10 23:07:52.404805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.404772 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:52.405085 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.405055 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.405085 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.405073 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/644ff901-6b0b-4676-b3ce-280ca8e34950-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:07:52.405085 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.405084 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.405286 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.405135 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-data" (OuterVolumeSpecName: "istio-data") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:52.405346 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.405293 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:52.406703 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.406680 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-token" (OuterVolumeSpecName: "istio-token") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:07:52.406830 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.406802 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-kube-api-access-djpwl" (OuterVolumeSpecName: "kube-api-access-djpwl") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "kube-api-access-djpwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:07:52.407095 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.407063 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:07:52.407095 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.407064 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "644ff901-6b0b-4676-b3ce-280ca8e34950" (UID: "644ff901-6b0b-4676-b3ce-280ca8e34950"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:07:52.506067 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506013 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/644ff901-6b0b-4676-b3ce-280ca8e34950-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.506067 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506059 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.506067 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506068 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.506067 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506078 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djpwl\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-kube-api-access-djpwl\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.506067 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506088 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.506436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506097 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.506436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.506105 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/644ff901-6b0b-4676-b3ce-280ca8e34950-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:07:52.892675 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.892626 2551 generic.go:356] "Generic (PLEG): container finished" podID="644ff901-6b0b-4676-b3ce-280ca8e34950" containerID="faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1" exitCode=0 Jul 10 23:07:52.892948 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.892682 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" Jul 10 23:07:52.892948 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.892685 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" event={"ID":"644ff901-6b0b-4676-b3ce-280ca8e34950","Type":"ContainerDied","Data":"faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1"} Jul 10 23:07:52.892948 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.892804 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2" event={"ID":"644ff901-6b0b-4676-b3ce-280ca8e34950","Type":"ContainerDied","Data":"1ad440def21d15118a5c0bfb5a91a8fb75e4ba197b83388c6440407058a8f220"} Jul 10 23:07:52.892948 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.892826 2551 scope.go:117] "RemoveContainer" containerID="faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1" Jul 10 23:07:52.903331 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.903308 2551 scope.go:117] "RemoveContainer" containerID="faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1" Jul 10 23:07:52.903664 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:07:52.903635 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1\": container with ID starting with faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1 not found: ID does not exist" containerID="faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1" Jul 10 23:07:52.903753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.903670 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1"} err="failed to get container status \"faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1\": rpc error: code = NotFound desc = could not find container \"faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1\": container with ID starting with faa3215854e02cfdf95c2a66dfdfa7c1669e27acf60b371c4fb337e7b69f9af1 not found: ID does not exist" Jul 10 23:07:52.916081 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.916046 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2"] Jul 10 23:07:52.928236 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:52.925523 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b379716f-openshift-default-7776fd587-tztm2"] Jul 10 23:07:53.016846 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:53.016803 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:53.122897 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:53.122855 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:53.480440 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:53.480400 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:53.579095 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:53.579058 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:53.677868 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:53.677826 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:07:54.042744 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:54.042704 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="644ff901-6b0b-4676-b3ce-280ca8e34950" path="/var/lib/kubelet/pods/644ff901-6b0b-4676-b3ce-280ca8e34950/volumes" Jul 10 23:07:55.096432 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:07:55.096386 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:01.089456 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:01.089420 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:01.195779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:01.195739 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:01.341195 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:01.341099 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:01.474704 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:01.474665 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:01.879353 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:01.879315 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:25.148515 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:25.148476 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:27.831772 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:27.831732 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:33.212591 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:33.212544 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:34.107352 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:34.107312 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:34.284525 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:34.284492 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:34.512822 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:34.512785 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:44.594751 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:44.594711 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:50.293754 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:50.293698 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:56.098933 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:56.098893 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:08:57.472444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:08:57.472400 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:06.817487 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:06.817453 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:07.633818 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:07.633788 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:13.392916 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:13.392876 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:13.497600 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:13.497567 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:13.595117 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:13.595080 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:13.695843 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:13.695736 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:14.425528 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:14.425490 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:14.518274 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:14.518236 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:14.613323 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:14.613284 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:16.166147 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:16.166109 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:17.203686 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:17.203646 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:09:21.763234 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:21.763169 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:21.821004 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:21.820971 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:22.461178 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:22.461143 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:26.596451 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:26.596408 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:09:28.495496 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.495459 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:28.593346 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.593308 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:28.723554 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.723520 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:28.997106 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.997067 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp"] Jul 10 23:09:28.997531 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.997512 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="644ff901-6b0b-4676-b3ce-280ca8e34950" containerName="istio-proxy" Jul 10 23:09:28.997531 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.997528 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="644ff901-6b0b-4676-b3ce-280ca8e34950" containerName="istio-proxy" Jul 10 23:09:28.997683 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:28.997633 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="644ff901-6b0b-4676-b3ce-280ca8e34950" containerName="istio-proxy" Jul 10 23:09:29.000834 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.000810 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.003451 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.003427 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-eb6601c6-openshift-default-dockercfg-kh7ff\"" Jul 10 23:09:29.021928 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.021890 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp"] Jul 10 23:09:29.024814 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.024784 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:29.119731 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119670 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-credential-socket\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.119966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119762 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-socket\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.119966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119781 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-istio-token\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.119966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119811 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-envoy\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.119966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119909 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/961db977-699f-46a8-a80b-b27b8bea78ec-istio-podinfo\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.119966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119941 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-certs\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.119966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119960 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-data\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.120317 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.119976 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jv6w\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-kube-api-access-8jv6w\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.120317 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.120024 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/961db977-699f-46a8-a80b-b27b8bea78ec-istiod-ca-cert\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.220883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.220845 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-socket\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.220883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.220888 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-istio-token\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.220910 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-envoy\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.220971 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/961db977-699f-46a8-a80b-b27b8bea78ec-istio-podinfo\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.220991 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-certs\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221012 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-data\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221039 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jv6w\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-kube-api-access-8jv6w\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221069 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/961db977-699f-46a8-a80b-b27b8bea78ec-istiod-ca-cert\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221132 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-credential-socket\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221540 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221347 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-socket\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221540 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221511 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-credential-socket\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221645 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221546 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-certs\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221645 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221612 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-data\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.221876 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.221852 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/961db977-699f-46a8-a80b-b27b8bea78ec-istiod-ca-cert\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.223568 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.223545 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-envoy\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.223810 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.223787 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/961db977-699f-46a8-a80b-b27b8bea78ec-istio-podinfo\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.229423 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.229383 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-istio-token\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.229676 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.229651 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jv6w\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-kube-api-access-8jv6w\") pod \"e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.290369 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.290327 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:29.312402 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.312365 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:29.472677 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.472642 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp"] Jul 10 23:09:29.475102 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:09:29.475071 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961db977_699f_46a8_a80b_b27b8bea78ec.slice/crio-f9a686ae8f9f6040fe0cbf0cf7ba4f8bf0cb96a07acff5ed33cef28a3a00a488 WatchSource:0}: Error finding container f9a686ae8f9f6040fe0cbf0cf7ba4f8bf0cb96a07acff5ed33cef28a3a00a488: Status 404 returned error can't find the container with id f9a686ae8f9f6040fe0cbf0cf7ba4f8bf0cb96a07acff5ed33cef28a3a00a488 Jul 10 23:09:29.477506 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.477474 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:09:29.477608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.477544 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:09:29.477608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:29.477571 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:09:30.387754 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:30.387713 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" event={"ID":"961db977-699f-46a8-a80b-b27b8bea78ec","Type":"ContainerStarted","Data":"0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211"} Jul 10 23:09:30.388198 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:30.387782 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" event={"ID":"961db977-699f-46a8-a80b-b27b8bea78ec","Type":"ContainerStarted","Data":"f9a686ae8f9f6040fe0cbf0cf7ba4f8bf0cb96a07acff5ed33cef28a3a00a488"} Jul 10 23:09:30.452619 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:30.452556 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" podStartSLOduration=2.452537549 podStartE2EDuration="2.452537549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:09:30.43038549 +0000 UTC m=+1629.067811168" watchObservedRunningTime="2026-07-10 23:09:30.452537549 +0000 UTC m=+1629.089963224" Jul 10 23:09:31.313501 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:31.313459 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:31.318628 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:31.318598 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:31.393473 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:31.393440 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:31.394682 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:31.394661 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:32.169792 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:32.169751 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:32.235315 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:32.235279 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:32.367506 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:32.367469 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:32.467036 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:32.466929 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:32.571120 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:32.571083 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:32.890624 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:32.890589 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:33.049649 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:33.049612 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:33.149526 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:33.149436 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:36.833646 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:36.833614 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:37.305600 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:37.305553 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:09:37.512109 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:37.512023 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:37.609508 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:37.609391 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:37.895584 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:37.895465 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:43.623772 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:43.623735 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:43.723255 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:43.723195 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:44.041490 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:44.041452 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:44.433809 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:44.433720 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:44.555593 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:44.555557 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:46.395313 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.395276 2551 patch_prober.go:28] interesting pod/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 23:09:46.395769 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.395333 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 23:09:46.474351 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.474319 2551 generic.go:356] "Generic (PLEG): container finished" podID="961db977-699f-46a8-a80b-b27b8bea78ec" containerID="0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211" exitCode=0 Jul 10 23:09:46.474532 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.474394 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" event={"ID":"961db977-699f-46a8-a80b-b27b8bea78ec","Type":"ContainerDied","Data":"0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211"} Jul 10 23:09:46.474838 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.474822 2551 scope.go:117] "RemoveContainer" containerID="0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211" Jul 10 23:09:46.475900 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.475872 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:09:46.475958 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.475941 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:09:46.475995 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:46.475985 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:09:47.484964 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:47.484921 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" event={"ID":"961db977-699f-46a8-a80b-b27b8bea78ec","Type":"ContainerStarted","Data":"67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e"} Jul 10 23:09:48.312688 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:48.312651 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:48.313021 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:48.312999 2551 patch_prober.go:28] interesting pod/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 23:09:48.313068 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:48.313049 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 23:09:49.113145 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:49.113104 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:09:49.313905 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:49.313861 2551 patch_prober.go:28] interesting pod/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 23:09:49.314080 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:49.313927 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 23:09:50.312817 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.312772 2551 patch_prober.go:28] interesting pod/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 23:09:50.313299 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.312831 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 23:09:50.624455 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.624346 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:50.689063 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.689011 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:50.904413 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.904320 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp"] Jul 10 23:09:50.904575 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.904521 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" containerID="cri-o://67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e" gracePeriod=30 Jul 10 23:09:50.953781 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:50.953737 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:51.129149 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:51.129112 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:51.244069 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:51.243971 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:51.340683 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:51.340649 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:51.444182 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:51.444147 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:56.791577 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.791505 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:56.887375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887337 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-istio-token\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887574 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887431 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-certs\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887596 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-envoy\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887629 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/961db977-699f-46a8-a80b-b27b8bea78ec-istio-podinfo\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887769 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887667 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-credential-socket\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887769 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887695 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-data\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887769 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887718 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/961db977-699f-46a8-a80b-b27b8bea78ec-istiod-ca-cert\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887769 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887729 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:09:56.887977 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887801 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-socket\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.887977 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.887835 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jv6w\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-kube-api-access-8jv6w\") pod \"961db977-699f-46a8-a80b-b27b8bea78ec\" (UID: \"961db977-699f-46a8-a80b-b27b8bea78ec\") " Jul 10 23:09:56.888080 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888008 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:09:56.888139 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888085 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:09:56.888139 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888109 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/961db977-699f-46a8-a80b-b27b8bea78ec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:09:56.888296 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888253 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.888296 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888271 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.888296 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888280 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.888296 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888289 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/961db977-699f-46a8-a80b-b27b8bea78ec-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.888709 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.888676 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-data" (OuterVolumeSpecName: "istio-data") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:09:56.889936 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.889903 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/961db977-699f-46a8-a80b-b27b8bea78ec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:09:56.890150 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.890127 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-kube-api-access-8jv6w" (OuterVolumeSpecName: "kube-api-access-8jv6w") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "kube-api-access-8jv6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:09:56.890268 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.890177 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-istio-token" (OuterVolumeSpecName: "istio-token") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:09:56.890268 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.890246 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "961db977-699f-46a8-a80b-b27b8bea78ec" (UID: "961db977-699f-46a8-a80b-b27b8bea78ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:09:56.989258 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.989180 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.989258 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.989248 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/961db977-699f-46a8-a80b-b27b8bea78ec-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.989258 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.989258 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/961db977-699f-46a8-a80b-b27b8bea78ec-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.989258 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.989269 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jv6w\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-kube-api-access-8jv6w\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:56.989566 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:56.989278 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/961db977-699f-46a8-a80b-b27b8bea78ec-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:09:57.018883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.018848 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:57.120265 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.120145 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:57.218482 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.218449 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:57.357849 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.357814 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:09:57.539703 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.539667 2551 generic.go:356] "Generic (PLEG): container finished" podID="961db977-699f-46a8-a80b-b27b8bea78ec" containerID="67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e" exitCode=0 Jul 10 23:09:57.539886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.539718 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" Jul 10 23:09:57.539886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.539756 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" event={"ID":"961db977-699f-46a8-a80b-b27b8bea78ec","Type":"ContainerDied","Data":"67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e"} Jul 10 23:09:57.539886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.539813 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp" event={"ID":"961db977-699f-46a8-a80b-b27b8bea78ec","Type":"ContainerDied","Data":"f9a686ae8f9f6040fe0cbf0cf7ba4f8bf0cb96a07acff5ed33cef28a3a00a488"} Jul 10 23:09:57.539886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.539844 2551 scope.go:117] "RemoveContainer" containerID="67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e" Jul 10 23:09:57.557567 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.557545 2551 scope.go:117] "RemoveContainer" containerID="0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211" Jul 10 23:09:57.566718 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.566695 2551 scope.go:117] "RemoveContainer" containerID="67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e" Jul 10 23:09:57.567049 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:09:57.567028 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e\": container with ID starting with 67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e not found: ID does not exist" containerID="67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e" Jul 10 23:09:57.567106 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.567058 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e"} err="failed to get container status \"67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e\": rpc error: code = NotFound desc = could not find container \"67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e\": container with ID starting with 67622395db01974a53ce36bdf737b3cd0a55902bc0ec09ee8751708dada76d4e not found: ID does not exist" Jul 10 23:09:57.567106 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.567078 2551 scope.go:117] "RemoveContainer" containerID="0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211" Jul 10 23:09:57.567226 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.567185 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp"] Jul 10 23:09:57.567400 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:09:57.567372 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211\": container with ID starting with 0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211 not found: ID does not exist" containerID="0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211" Jul 10 23:09:57.567447 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.567407 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211"} err="failed to get container status \"0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211\": rpc error: code = NotFound desc = could not find container \"0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211\": container with ID starting with 0eb3a1922a62e6894d8a08e9228587b768b66104fec9bbca01a931b07855f211 not found: ID does not exist" Jul 10 23:09:57.571355 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:57.571322 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-eb6601c6-openshift-default-5dc5c46d59-p48mp"] Jul 10 23:09:58.042720 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:09:58.042683 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" path="/var/lib/kubelet/pods/961db977-699f-46a8-a80b-b27b8bea78ec/volumes" Jul 10 23:10:03.305076 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:03.305045 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:03.405060 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:03.405022 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:10.007399 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.007359 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:10.263722 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.263621 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf"] Jul 10 23:10:10.264074 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.264058 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" Jul 10 23:10:10.264145 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.264077 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" Jul 10 23:10:10.264225 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.264171 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" Jul 10 23:10:10.269195 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.269164 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.272136 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.272112 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ea0b66f6-openshift-default-dockercfg-zwknv\"" Jul 10 23:10:10.283871 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.283843 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf"] Jul 10 23:10:10.300983 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.300944 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:10.314534 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314502 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgbvp\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-kube-api-access-tgbvp\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314534 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314539 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-data\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314584 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-credential-socket\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314618 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-certs\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314638 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istiod-ca-cert\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314666 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-socket\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314699 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-envoy\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.314779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314715 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-token\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.315039 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.314815 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-podinfo\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416110 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416067 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-podinfo\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416368 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416127 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgbvp\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-kube-api-access-tgbvp\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416368 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416277 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-data\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416368 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416344 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-credential-socket\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416553 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416400 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-certs\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416553 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416425 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istiod-ca-cert\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416553 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416455 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-socket\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416553 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416526 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-envoy\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416819 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416555 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-token\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416831 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-credential-socket\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416857 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-certs\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.416995 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.416970 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-data\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.417138 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.417112 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-socket\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.417138 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.417134 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istiod-ca-cert\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.418831 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.418809 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-envoy\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.419006 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.418984 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-podinfo\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.425399 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.425364 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-token\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.425612 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.425587 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgbvp\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-kube-api-access-tgbvp\") pod \"e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.557458 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.557366 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:10.581463 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.581414 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:10.738890 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.738861 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf"] Jul 10 23:10:10.741191 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:10:10.741164 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb660e5_a719_4d2c_a745_f86c8c17a0a9.slice/crio-4b0e766a5263772108b86f44684f554ff0bbd6355f6eb8d5cbd207ee608cd384 WatchSource:0}: Error finding container 4b0e766a5263772108b86f44684f554ff0bbd6355f6eb8d5cbd207ee608cd384: Status 404 returned error can't find the container with id 4b0e766a5263772108b86f44684f554ff0bbd6355f6eb8d5cbd207ee608cd384 Jul 10 23:10:10.743338 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.743299 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:10:10.743443 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.743386 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:10:10.743443 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:10.743417 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:10:11.616708 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:11.616666 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" event={"ID":"4cb660e5-a719-4d2c-a745-f86c8c17a0a9","Type":"ContainerStarted","Data":"e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580"} Jul 10 23:10:11.617105 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:11.616732 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" event={"ID":"4cb660e5-a719-4d2c-a745-f86c8c17a0a9","Type":"ContainerStarted","Data":"4b0e766a5263772108b86f44684f554ff0bbd6355f6eb8d5cbd207ee608cd384"} Jul 10 23:10:11.637163 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:11.637109 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podStartSLOduration=1.637087063 podStartE2EDuration="1.637087063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:10:11.635803144 +0000 UTC m=+1670.273228845" watchObservedRunningTime="2026-07-10 23:10:11.637087063 +0000 UTC m=+1670.274512739" Jul 10 23:10:12.582653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:12.582613 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:12.587781 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:12.587750 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:12.622744 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:12.622710 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:12.623846 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:12.623826 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:13.592778 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:13.592740 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:13.725793 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:13.725762 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:13.868719 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:13.868619 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:13.965711 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:13.965668 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:14.282484 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:14.282434 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:14.382651 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:14.382604 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:18.063840 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:18.063800 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:19.992859 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:19.992820 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:10:28.010264 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.008712 2551 patch_prober.go:28] interesting pod/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:42746->10.132.0.52:15021: read: connection reset by peer" start-of-body= Jul 10 23:10:28.010264 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.008785 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:42746->10.132.0.52:15021: read: connection reset by peer" Jul 10 23:10:28.718514 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.718481 2551 generic.go:356] "Generic (PLEG): container finished" podID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerID="e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580" exitCode=0 Jul 10 23:10:28.718690 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.718551 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" event={"ID":"4cb660e5-a719-4d2c-a745-f86c8c17a0a9","Type":"ContainerDied","Data":"e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580"} Jul 10 23:10:28.718980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.718965 2551 scope.go:117] "RemoveContainer" containerID="e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580" Jul 10 23:10:28.719982 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.719956 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:10:28.726033 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.725556 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:10:28.726033 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:28.725706 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:10:29.726111 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:29.726074 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" event={"ID":"4cb660e5-a719-4d2c-a745-f86c8c17a0a9","Type":"ContainerStarted","Data":"fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73"} Jul 10 23:10:30.582250 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:30.582196 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:30.583109 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:30.583077 2551 patch_prober.go:28] interesting pod/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 23:10:30.583690 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:30.583335 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 23:10:31.583094 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:31.582614 2551 patch_prober.go:28] interesting pod/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 23:10:31.583094 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:31.582685 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 23:10:32.581943 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:32.581891 2551 patch_prober.go:28] interesting pod/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 23:10:32.582167 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:32.581965 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 23:10:33.582382 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:33.582335 2551 patch_prober.go:28] interesting pod/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 23:10:33.583236 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:33.583178 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 23:10:34.586384 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:34.586342 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:34.587021 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:34.586668 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:34.587849 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:34.587826 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:10:38.833975 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:38.833938 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:38.934729 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:38.934693 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:43.216026 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:43.215994 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:10:46.399380 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:10:46.399279 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:11:00.899779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:00.899733 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:11:05.800529 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:05.800493 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:06.244892 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:06.244798 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:06.344415 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:06.344370 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:06.486849 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:06.486813 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:10.915970 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:10.915927 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:11:11.876674 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:11.876623 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:16.913902 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:16.913867 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:20.888929 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:20.888881 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:11:53.956677 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:53.956634 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:54.115886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:54.115850 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:56.094387 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:56.094346 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:11:59.670608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:59.670568 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:59.772867 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:59.772834 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:11:59.941521 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:59.941423 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf"] Jul 10 23:11:59.941716 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:59.941681 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" containerID="cri-o://fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73" gracePeriod=30 Jul 10 23:11:59.970022 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:11:59.969986 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:00.177296 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:00.177256 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:01.202387 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:01.202354 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:01.301779 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:01.301722 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:01.400241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:01.400184 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:02.146762 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:02.146730 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:02.244833 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:02.244774 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:02.345144 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:02.345113 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:03.675579 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:03.675543 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:04.588980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:04.588941 2551 patch_prober.go:28] interesting pod/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 23:12:04.589187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:04.589073 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 23:12:05.219627 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.219550 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:12:05.227635 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.227598 2551 generic.go:356] "Generic (PLEG): container finished" podID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerID="fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73" exitCode=0 Jul 10 23:12:05.227838 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.227655 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" Jul 10 23:12:05.227838 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.227676 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" event={"ID":"4cb660e5-a719-4d2c-a745-f86c8c17a0a9","Type":"ContainerDied","Data":"fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73"} Jul 10 23:12:05.227838 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.227758 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf" event={"ID":"4cb660e5-a719-4d2c-a745-f86c8c17a0a9","Type":"ContainerDied","Data":"4b0e766a5263772108b86f44684f554ff0bbd6355f6eb8d5cbd207ee608cd384"} Jul 10 23:12:05.227838 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.227781 2551 scope.go:117] "RemoveContainer" containerID="fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73" Jul 10 23:12:05.237179 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.237153 2551 scope.go:117] "RemoveContainer" containerID="e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580" Jul 10 23:12:05.248337 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.248314 2551 scope.go:117] "RemoveContainer" containerID="fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73" Jul 10 23:12:05.248714 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:12:05.248684 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73\": container with ID starting with fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73 not found: ID does not exist" containerID="fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73" Jul 10 23:12:05.248817 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.248728 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73"} err="failed to get container status \"fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73\": rpc error: code = NotFound desc = could not find container \"fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73\": container with ID starting with fe413c8c4d968f9cbfdb678ce6445942e6db636a3517fb3dfc0b94a72de86f73 not found: ID does not exist" Jul 10 23:12:05.248817 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.248756 2551 scope.go:117] "RemoveContainer" containerID="e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580" Jul 10 23:12:05.249043 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:12:05.249020 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580\": container with ID starting with e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580 not found: ID does not exist" containerID="e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580" Jul 10 23:12:05.249118 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.249050 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580"} err="failed to get container status \"e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580\": rpc error: code = NotFound desc = could not find container \"e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580\": container with ID starting with e04c4adba0a4251cb8bc78d3a6a3af8511b3c0c27c3bca60f0f408e17ee99580 not found: ID does not exist" Jul 10 23:12:05.325530 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325481 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-envoy\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.325530 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325536 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-certs\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.325776 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325582 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgbvp\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-kube-api-access-tgbvp\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.325776 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325599 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-data\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.325776 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325618 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-podinfo\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.325776 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325688 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istiod-ca-cert\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.325776 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325711 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-token\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.326127 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325783 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-credential-socket\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.326127 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325808 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-socket\") pod \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\" (UID: \"4cb660e5-a719-4d2c-a745-f86c8c17a0a9\") " Jul 10 23:12:05.326127 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.325941 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:12:05.326316 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.326142 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:12:05.326316 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.326167 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.326426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.326385 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:12:05.326689 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.326664 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:12:05.326786 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.326747 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:12:05.327943 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.327920 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:12:05.328019 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.327973 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-kube-api-access-tgbvp" (OuterVolumeSpecName: "kube-api-access-tgbvp") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "kube-api-access-tgbvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:12:05.328427 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.328403 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:12:05.328497 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.328418 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4cb660e5-a719-4d2c-a745-f86c8c17a0a9" (UID: "4cb660e5-a719-4d2c-a745-f86c8c17a0a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:12:05.427303 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427256 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427303 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427295 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgbvp\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-kube-api-access-tgbvp\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427303 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427309 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427577 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427321 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427577 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427332 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427577 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427343 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427577 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427354 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.427577 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.427366 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4cb660e5-a719-4d2c-a745-f86c8c17a0a9-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:12:05.550338 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.550301 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf"] Jul 10 23:12:05.553253 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:05.553205 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ea0b66f6-openshift-default-59b447c65f-5bcmf"] Jul 10 23:12:06.004641 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:06.004593 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:12:06.044005 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:06.043966 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" path="/var/lib/kubelet/pods/4cb660e5-a719-4d2c-a745-f86c8c17a0a9/volumes" Jul 10 23:12:09.678462 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:09.678410 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:09.778479 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:09.778445 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:10.353189 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:10.353152 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:14.801714 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:14.801607 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:12:16.347055 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:16.346992 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:16.450854 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:16.450820 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:16.543287 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:16.543247 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:22.108227 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:22.108183 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:12:22.108988 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:22.108968 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:12:27.027454 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:27.027420 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:27.427135 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:27.427001 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:48.329904 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:48.329863 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:54.461632 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:54.461592 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:54.563041 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:54.563006 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:54.805436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:54.805406 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:55.098222 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098123 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s"] Jul 10 23:12:55.098604 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098588 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" Jul 10 23:12:55.098661 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098606 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" Jul 10 23:12:55.098661 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098616 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" Jul 10 23:12:55.098661 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098622 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" Jul 10 23:12:55.098766 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098695 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" Jul 10 23:12:55.098766 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098705 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="961db977-699f-46a8-a80b-b27b8bea78ec" containerName="istio-proxy" Jul 10 23:12:55.098766 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.098711 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" Jul 10 23:12:55.101918 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.101898 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.104555 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.104518 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1dc3cf-openshift-default-dockercfg-gbn6d\"" Jul 10 23:12:55.119969 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.119936 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s"] Jul 10 23:12:55.128244 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.128191 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:55.193253 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193182 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-socket\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193459 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193260 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-credential-socket\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193459 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193286 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-podinfo\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193459 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193302 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qp9w\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-kube-api-access-7qp9w\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193459 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193383 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-data\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193459 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193443 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-envoy\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193732 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193497 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-token\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193732 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193541 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edab21c2-5ca9-4927-ad99-b28bce836cc8-istiod-ca-cert\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.193732 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.193589 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-certs\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.283467 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.283433 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:55.294692 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.294653 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-data\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.294692 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.294694 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-envoy\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.294877 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.294823 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-token\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.294877 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.294867 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edab21c2-5ca9-4927-ad99-b28bce836cc8-istiod-ca-cert\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.294954 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.294917 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-certs\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295003 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.294984 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-socket\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295052 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295007 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-credential-socket\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295052 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295035 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-podinfo\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295159 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295061 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qp9w\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-kube-api-access-7qp9w\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295159 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295084 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-data\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295448 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295423 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-socket\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295541 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295440 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-certs\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295632 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295608 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-credential-socket\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.295724 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.295706 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edab21c2-5ca9-4927-ad99-b28bce836cc8-istiod-ca-cert\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.297129 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.297104 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-envoy\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.297692 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.297671 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-podinfo\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.303634 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.303607 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qp9w\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-kube-api-access-7qp9w\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.303883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.303862 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-token\") pod \"e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.413806 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.413691 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:55.581840 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.581620 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s"] Jul 10 23:12:55.585146 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:12:55.585115 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedab21c2_5ca9_4927_ad99_b28bce836cc8.slice/crio-49343c39938c4efb08697310ce1e13ed8cdf057c3a6393f2f494b01ae44c8318 WatchSource:0}: Error finding container 49343c39938c4efb08697310ce1e13ed8cdf057c3a6393f2f494b01ae44c8318: Status 404 returned error can't find the container with id 49343c39938c4efb08697310ce1e13ed8cdf057c3a6393f2f494b01ae44c8318 Jul 10 23:12:55.587645 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.587609 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:12:55.587753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.587696 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:12:55.587753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:55.587742 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:12:56.506719 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:56.506682 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" event={"ID":"edab21c2-5ca9-4927-ad99-b28bce836cc8","Type":"ContainerStarted","Data":"696cde3e7950c87a2e503496de9c69ad0b626db152c4e9af6cb333e9bfa53d95"} Jul 10 23:12:56.506916 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:56.506736 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" event={"ID":"edab21c2-5ca9-4927-ad99-b28bce836cc8","Type":"ContainerStarted","Data":"49343c39938c4efb08697310ce1e13ed8cdf057c3a6393f2f494b01ae44c8318"} Jul 10 23:12:56.530198 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:56.530136 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podStartSLOduration=1.53011703 podStartE2EDuration="1.53011703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:12:56.527454785 +0000 UTC m=+1835.164880472" watchObservedRunningTime="2026-07-10 23:12:56.53011703 +0000 UTC m=+1835.167542704" Jul 10 23:12:57.414899 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:57.414858 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:57.420752 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:57.420722 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:57.511945 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:57.511908 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:57.512825 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:57.512802 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:12:58.452663 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:58.452628 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:58.531391 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:58.531343 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:58.597653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:58.597610 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:58.755142 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:58.755112 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:59.012006 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:59.011916 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:12:59.117700 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:12:59.117663 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:02.952970 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:02.952930 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:03.260399 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.260359 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:03.540350 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.540201 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh"] Jul 10 23:13:03.540964 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.540944 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" Jul 10 23:13:03.540964 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.540966 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb660e5-a719-4d2c-a745-f86c8c17a0a9" containerName="istio-proxy" Jul 10 23:13:03.544944 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.544914 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.547776 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.547745 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-51f2b4-openshift-default-dockercfg-hdzw5\"" Jul 10 23:13:03.558555 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.558523 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh"] Jul 10 23:13:03.569960 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.569922 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:03.675605 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.675568 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:03.680260 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680225 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-credential-socket\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680392 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680269 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-certs\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680392 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680335 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-token\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680392 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680367 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vz2b\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-kube-api-access-8vz2b\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680521 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680393 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-socket\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680521 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680423 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-data\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680521 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680438 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-podinfo\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680521 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680473 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istiod-ca-cert\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.680521 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.680510 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-envoy\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781568 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vz2b\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-kube-api-access-8vz2b\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781619 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-socket\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781896 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781664 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-data\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781896 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781686 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-podinfo\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781896 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781716 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istiod-ca-cert\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781896 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781762 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-envoy\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781896 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781832 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-credential-socket\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.781896 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781860 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-certs\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.782201 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.781931 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-token\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.782201 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.782156 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-socket\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.782365 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.782289 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-credential-socket\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.782365 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.782340 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-certs\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.782446 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.782359 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-data\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.782609 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.782588 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istiod-ca-cert\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.784340 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.784306 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-envoy\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.784593 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.784575 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-podinfo\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.789307 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.789278 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-token\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.789463 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.789360 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vz2b\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-kube-api-access-8vz2b\") pod \"e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:03.859491 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:03.859386 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:04.021009 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.020971 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh"] Jul 10 23:13:04.022997 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:13:04.022953 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e03011b_e0bf_4c6e_ae76_d602ae68e256.slice/crio-04f4695ac3324469c7a7d4f4a7cb6f3fabd4bcf2e530019345d21b71913ae4ae WatchSource:0}: Error finding container 04f4695ac3324469c7a7d4f4a7cb6f3fabd4bcf2e530019345d21b71913ae4ae: Status 404 returned error can't find the container with id 04f4695ac3324469c7a7d4f4a7cb6f3fabd4bcf2e530019345d21b71913ae4ae Jul 10 23:13:04.025350 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.025309 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:04.025452 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.025394 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:04.025452 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.025423 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:04.553194 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.553150 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" event={"ID":"9e03011b-e0bf-4c6e-ae76-d602ae68e256","Type":"ContainerStarted","Data":"8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942"} Jul 10 23:13:04.553447 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.553258 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" event={"ID":"9e03011b-e0bf-4c6e-ae76-d602ae68e256","Type":"ContainerStarted","Data":"04f4695ac3324469c7a7d4f4a7cb6f3fabd4bcf2e530019345d21b71913ae4ae"} Jul 10 23:13:04.608724 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.608656 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podStartSLOduration=1.608631339 podStartE2EDuration="1.608631339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:13:04.607253036 +0000 UTC m=+1843.244678712" watchObservedRunningTime="2026-07-10 23:13:04.608631339 +0000 UTC m=+1843.246056994" Jul 10 23:13:04.860071 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:04.859972 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:05.865375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:05.865341 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:06.503495 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:06.503455 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:06.564606 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:06.564576 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:06.565815 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:06.565787 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:06.582268 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:06.582205 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:06.623164 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:06.623125 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:13:06.647367 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:06.647334 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:07.154768 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:07.154729 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:07.250167 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:07.250134 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:10.989078 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:10.989040 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:13.553425 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:13.553378 2551 patch_prober.go:28] interesting pod/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 23:13:13.553865 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:13.553449 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 23:13:15.622198 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:15.622160 2551 generic.go:356] "Generic (PLEG): container finished" podID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerID="696cde3e7950c87a2e503496de9c69ad0b626db152c4e9af6cb333e9bfa53d95" exitCode=0 Jul 10 23:13:15.622681 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:15.622239 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" event={"ID":"edab21c2-5ca9-4927-ad99-b28bce836cc8","Type":"ContainerDied","Data":"696cde3e7950c87a2e503496de9c69ad0b626db152c4e9af6cb333e9bfa53d95"} Jul 10 23:13:15.622681 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:15.622675 2551 scope.go:117] "RemoveContainer" containerID="696cde3e7950c87a2e503496de9c69ad0b626db152c4e9af6cb333e9bfa53d95" Jul 10 23:13:15.623853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:15.623819 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:15.623983 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:15.623913 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:15.623983 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:15.623961 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:16.629886 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:16.629846 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" event={"ID":"edab21c2-5ca9-4927-ad99-b28bce836cc8","Type":"ContainerStarted","Data":"cade1e71db93f90132734d9c6b44cb47d8430368302dc25ecd5418a0e881f0e2"} Jul 10 23:13:17.414020 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:17.413947 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:13:17.414542 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:17.414506 2551 patch_prober.go:28] interesting pod/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 23:13:17.414673 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:17.414585 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 23:13:17.696878 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:17.696758 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:13:18.414442 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:18.414391 2551 patch_prober.go:28] interesting pod/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 23:13:18.414721 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:18.414459 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 23:13:19.414225 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:19.414176 2551 patch_prober.go:28] interesting pod/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 23:13:19.414644 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:19.414248 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 23:13:19.987504 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:19.987450 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:20.133636 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:20.133592 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:20.414369 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:20.414330 2551 patch_prober.go:28] interesting pod/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 23:13:20.414895 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:20.414404 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 23:13:21.418383 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:21.418332 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:13:21.418869 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:21.418847 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:13:21.419796 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:21.419773 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:13:21.874152 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:21.873762 2551 patch_prober.go:28] interesting pod/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:54814->10.132.0.54:15021: read: connection reset by peer" start-of-body= Jul 10 23:13:21.874152 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:21.873839 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:54814->10.132.0.54:15021: read: connection reset by peer" Jul 10 23:13:22.662421 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:22.662384 2551 generic.go:356] "Generic (PLEG): container finished" podID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerID="8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942" exitCode=0 Jul 10 23:13:22.662854 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:22.662454 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" event={"ID":"9e03011b-e0bf-4c6e-ae76-d602ae68e256","Type":"ContainerDied","Data":"8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942"} Jul 10 23:13:22.663066 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:22.663044 2551 scope.go:117] "RemoveContainer" containerID="8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942" Jul 10 23:13:22.664107 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:22.664076 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:22.664180 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:22.664167 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:22.664233 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:22.664202 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:13:23.671554 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:23.671506 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" event={"ID":"9e03011b-e0bf-4c6e-ae76-d602ae68e256","Type":"ContainerStarted","Data":"3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6"} Jul 10 23:13:23.860283 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:23.860243 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:23.860630 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:23.860598 2551 patch_prober.go:28] interesting pod/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 23:13:23.860689 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:23.860657 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 23:13:24.860291 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:24.860204 2551 patch_prober.go:28] interesting pod/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 23:13:24.860744 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:24.860338 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 23:13:25.860192 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:25.860144 2551 patch_prober.go:28] interesting pod/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 23:13:25.860428 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:25.860250 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 23:13:26.860509 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:26.860466 2551 patch_prober.go:28] interesting pod/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 23:13:26.861030 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:26.860530 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 23:13:27.863988 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:27.863954 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:27.864421 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:27.864238 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:27.865114 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:27.865093 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:13:27.953252 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:27.953181 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:28.329528 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:28.329486 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:28.431621 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:28.431581 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:13:31.104168 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:31.104129 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:13:41.803541 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:41.803494 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:13:57.353878 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:13:57.353786 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:02.831245 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:02.831183 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:14:24.888456 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:24.888421 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:25.847833 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:25.847801 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:26.105411 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:26.105317 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:26.463341 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:26.463244 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:26.563376 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:26.563340 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:26.663978 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:26.663938 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:26.815572 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:26.815532 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:14:29.706195 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:29.706153 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:14:31.542883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:14:31.542842 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:08.807078 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:08.807038 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:09.065339 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.065240 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s"] Jul 10 23:15:09.065563 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.065532 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" containerID="cri-o://cade1e71db93f90132734d9c6b44cb47d8430368302dc25ecd5418a0e881f0e2" gracePeriod=30 Jul 10 23:15:09.087844 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.087802 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:09.301143 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.301107 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:09.455477 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.455387 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:09.852601 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.852564 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:09.950610 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:09.950564 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:10.056324 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:10.056286 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:10.164728 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:10.164602 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:13.090263 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:13.090204 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:15:14.285758 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.285724 2551 generic.go:356] "Generic (PLEG): container finished" podID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerID="cade1e71db93f90132734d9c6b44cb47d8430368302dc25ecd5418a0e881f0e2" exitCode=0 Jul 10 23:15:14.286190 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.285805 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" event={"ID":"edab21c2-5ca9-4927-ad99-b28bce836cc8","Type":"ContainerDied","Data":"cade1e71db93f90132734d9c6b44cb47d8430368302dc25ecd5418a0e881f0e2"} Jul 10 23:15:14.286190 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.285879 2551 scope.go:117] "RemoveContainer" containerID="696cde3e7950c87a2e503496de9c69ad0b626db152c4e9af6cb333e9bfa53d95" Jul 10 23:15:14.346644 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.346608 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:15:14.500374 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500316 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qp9w\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-kube-api-access-7qp9w\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500374 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500373 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-token\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500405 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-podinfo\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500450 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-certs\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500497 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-data\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500562 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-socket\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500587 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-envoy\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500659 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500633 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-credential-socket\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500959 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500663 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edab21c2-5ca9-4927-ad99-b28bce836cc8-istiod-ca-cert\") pod \"edab21c2-5ca9-4927-ad99-b28bce836cc8\" (UID: \"edab21c2-5ca9-4927-ad99-b28bce836cc8\") " Jul 10 23:15:14.500959 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.500857 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:14.501354 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.501104 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.501354 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.501194 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:14.501354 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.501244 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edab21c2-5ca9-4927-ad99-b28bce836cc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:15:14.501354 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.501323 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:14.501629 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.501551 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:14.502944 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.502919 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-kube-api-access-7qp9w" (OuterVolumeSpecName: "kube-api-access-7qp9w") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "kube-api-access-7qp9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:14.503086 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.503061 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:15:14.503151 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.503072 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:14.503519 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.503493 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "edab21c2-5ca9-4927-ad99-b28bce836cc8" (UID: "edab21c2-5ca9-4927-ad99-b28bce836cc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602087 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qp9w\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-kube-api-access-7qp9w\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602126 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602136 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602144 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602154 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602163 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602171 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edab21c2-5ca9-4927-ad99-b28bce836cc8-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:14.602187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:14.602179 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edab21c2-5ca9-4927-ad99-b28bce836cc8-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:15.293101 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.293070 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" Jul 10 23:15:15.293101 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.293096 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s" event={"ID":"edab21c2-5ca9-4927-ad99-b28bce836cc8","Type":"ContainerDied","Data":"49343c39938c4efb08697310ce1e13ed8cdf057c3a6393f2f494b01ae44c8318"} Jul 10 23:15:15.293667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.293153 2551 scope.go:117] "RemoveContainer" containerID="cade1e71db93f90132734d9c6b44cb47d8430368302dc25ecd5418a0e881f0e2" Jul 10 23:15:15.316145 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.316111 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s"] Jul 10 23:15:15.319529 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.319498 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1dc3cf-openshift-default-5f4d649557-fw86s"] Jul 10 23:15:15.480265 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.480227 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:15.581159 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.581079 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:15.723841 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.723797 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh"] Jul 10 23:15:15.724047 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.724022 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" containerID="cri-o://3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6" gracePeriod=30 Jul 10 23:15:15.748483 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.748445 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:15.984547 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:15.984456 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:16.043534 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.043492 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" path="/var/lib/kubelet/pods/edab21c2-5ca9-4927-ad99-b28bce836cc8/volumes" Jul 10 23:15:16.316821 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.316787 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:16.620004 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.619899 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs"] Jul 10 23:15:16.620565 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.620543 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" Jul 10 23:15:16.620565 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.620564 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" Jul 10 23:15:16.620766 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.620577 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" Jul 10 23:15:16.620766 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.620587 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" Jul 10 23:15:16.620766 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.620717 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" Jul 10 23:15:16.625874 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.625842 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.628561 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.628534 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d639f4-openshift-default-dockercfg-lwctl\"" Jul 10 23:15:16.635642 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.635608 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs"] Jul 10 23:15:16.648322 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.648283 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:16.818319 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818281 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-data\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818527 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818330 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-credential-socket\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818527 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818350 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-podinfo\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818527 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818463 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-certs\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818527 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818517 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-socket\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818683 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818576 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-envoy\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818683 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818627 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chhs2\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-kube-api-access-chhs2\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818770 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818685 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istiod-ca-cert\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.818770 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.818717 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-token\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.855419 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.855376 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:16.919550 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919450 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-certs\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919550 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919499 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-socket\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919749 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919561 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-envoy\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919749 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919701 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chhs2\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-kube-api-access-chhs2\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919835 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istiod-ca-cert\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919891 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919831 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-token\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919941 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919888 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-certs\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.919941 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919895 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-data\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.920049 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919946 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-socket\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.920049 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919949 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-credential-socket\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.920049 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.919990 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-podinfo\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.920231 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.920161 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-data\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.920301 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.920270 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-credential-socket\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.920569 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.920546 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istiod-ca-cert\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.922095 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.922068 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-envoy\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.922412 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.922392 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-podinfo\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.929891 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.929855 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-token\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.930160 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.930135 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhs2\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-kube-api-access-chhs2\") pod \"e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:16.939898 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:16.939866 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:17.098910 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.098856 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs"] Jul 10 23:15:17.103032 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:15:17.102999 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f7c5dd_0810_4b1d_b745_aacde8005e45.slice/crio-ffbee129b31775903d3c37988087ea16c18f86fdeb0edb40da5499796d49ba76 WatchSource:0}: Error finding container ffbee129b31775903d3c37988087ea16c18f86fdeb0edb40da5499796d49ba76: Status 404 returned error can't find the container with id ffbee129b31775903d3c37988087ea16c18f86fdeb0edb40da5499796d49ba76 Jul 10 23:15:17.105284 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.105249 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:17.105369 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.105329 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:17.105416 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.105379 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:17.308728 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.308684 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" event={"ID":"c7f7c5dd-0810-4b1d-b745-aacde8005e45","Type":"ContainerStarted","Data":"9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080"} Jul 10 23:15:17.308951 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.308775 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" event={"ID":"c7f7c5dd-0810-4b1d-b745-aacde8005e45","Type":"ContainerStarted","Data":"ffbee129b31775903d3c37988087ea16c18f86fdeb0edb40da5499796d49ba76"} Jul 10 23:15:17.337469 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.337404 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podStartSLOduration=1.337381132 podStartE2EDuration="1.337381132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:15:17.335316319 +0000 UTC m=+1975.972741995" watchObservedRunningTime="2026-07-10 23:15:17.337381132 +0000 UTC m=+1975.974806811" Jul 10 23:15:17.940522 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:17.940474 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:18.946242 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:18.946190 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:19.229416 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.229322 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:19.309097 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.309059 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:19.320057 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.320025 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:19.321268 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.321248 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:19.381781 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.381744 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:19.487905 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.487820 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:19.903134 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:19.903101 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:20.007957 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:20.007922 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:21.006590 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.006555 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:15:21.051876 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.051840 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-data\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.051886 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-podinfo\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.051908 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-token\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.051928 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istiod-ca-cert\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052117 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-credential-socket\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052226 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-envoy\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052290 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-socket\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052336 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vz2b\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-kube-api-access-8vz2b\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052383 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-certs\") pod \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\" (UID: \"9e03011b-e0bf-4c6e-ae76-d602ae68e256\") " Jul 10 23:15:21.052843 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052765 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:21.053121 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052928 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:21.053121 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.052941 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:21.053500 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.053410 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:15:21.053500 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.053475 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:21.054726 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.054696 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:21.055382 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.055355 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:21.055498 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.055418 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:15:21.055498 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.055427 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-kube-api-access-8vz2b" (OuterVolumeSpecName: "kube-api-access-8vz2b") pod "9e03011b-e0bf-4c6e-ae76-d602ae68e256" (UID: "9e03011b-e0bf-4c6e-ae76-d602ae68e256"). InnerVolumeSpecName "kube-api-access-8vz2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:21.153516 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153472 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153516 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153508 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vz2b\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-kube-api-access-8vz2b\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153516 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153520 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153516 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153529 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153844 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153537 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153844 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153545 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153844 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153553 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153844 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153560 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.153844 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.153567 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03011b-e0bf-4c6e-ae76-d602ae68e256-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:21.334022 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.333983 2551 generic.go:356] "Generic (PLEG): container finished" podID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerID="3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6" exitCode=0 Jul 10 23:15:21.334022 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.334037 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" Jul 10 23:15:21.334289 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.334061 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" event={"ID":"9e03011b-e0bf-4c6e-ae76-d602ae68e256","Type":"ContainerDied","Data":"3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6"} Jul 10 23:15:21.334289 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.334126 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh" event={"ID":"9e03011b-e0bf-4c6e-ae76-d602ae68e256","Type":"ContainerDied","Data":"04f4695ac3324469c7a7d4f4a7cb6f3fabd4bcf2e530019345d21b71913ae4ae"} Jul 10 23:15:21.334289 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.334150 2551 scope.go:117] "RemoveContainer" containerID="3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6" Jul 10 23:15:21.345162 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.345138 2551 scope.go:117] "RemoveContainer" containerID="8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942" Jul 10 23:15:21.354737 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.354713 2551 scope.go:117] "RemoveContainer" containerID="3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6" Jul 10 23:15:21.355121 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:15:21.355090 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6\": container with ID starting with 3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6 not found: ID does not exist" containerID="3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6" Jul 10 23:15:21.355249 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.355136 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6"} err="failed to get container status \"3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6\": rpc error: code = NotFound desc = could not find container \"3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6\": container with ID starting with 3c3f634f51779dc11f0cb5ed19f15b180ba065df23aa2f0ea231f08c454d95a6 not found: ID does not exist" Jul 10 23:15:21.355249 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.355166 2551 scope.go:117] "RemoveContainer" containerID="8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942" Jul 10 23:15:21.355730 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:15:21.355652 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942\": container with ID starting with 8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942 not found: ID does not exist" containerID="8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942" Jul 10 23:15:21.355869 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.355698 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942"} err="failed to get container status \"8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942\": rpc error: code = NotFound desc = could not find container \"8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942\": container with ID starting with 8441a7a7aa4d80a84af416a422c67bc0fd4bb3145461b10a92893d9212ef4942 not found: ID does not exist" Jul 10 23:15:21.357932 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.357908 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh"] Jul 10 23:15:21.362149 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:21.362118 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-51f2b4-openshift-default-686f57778c-f5nxh"] Jul 10 23:15:22.042568 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:22.042526 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" path="/var/lib/kubelet/pods/9e03011b-e0bf-4c6e-ae76-d602ae68e256/volumes" Jul 10 23:15:23.720588 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:23.720549 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:24.219178 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.219128 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:24.550009 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.549971 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g"] Jul 10 23:15:24.550417 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.550403 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" Jul 10 23:15:24.550468 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.550419 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" Jul 10 23:15:24.550511 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.550500 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="edab21c2-5ca9-4927-ad99-b28bce836cc8" containerName="istio-proxy" Jul 10 23:15:24.550511 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.550509 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" Jul 10 23:15:24.550579 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.550517 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" Jul 10 23:15:24.555466 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.555438 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.558092 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.558064 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8b0687-openshift-default-dockercfg-knl7t\"" Jul 10 23:15:24.584162 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584115 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-envoy\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584162 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584160 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5029fa33-453d-46cf-b6b3-44ea0568bad2-istiod-ca-cert\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584449 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584257 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-credential-socket\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584449 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584289 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-certs\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584449 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584312 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjkxz\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-kube-api-access-mjkxz\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584449 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584391 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-podinfo\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584512 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-data\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584572 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-token\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.584667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584585 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g"] Jul 10 23:15:24.584667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.584602 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-socket\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.617242 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.617187 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:24.685734 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.685691 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-token\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.685734 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.685735 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-socket\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686010 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.685777 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-envoy\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686010 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.685800 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5029fa33-453d-46cf-b6b3-44ea0568bad2-istiod-ca-cert\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686010 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.685843 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-credential-socket\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686010 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.685872 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-certs\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686238 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686093 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjkxz\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-kube-api-access-mjkxz\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686238 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686157 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-podinfo\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686343 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686293 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-socket\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686401 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686338 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-credential-socket\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686401 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686384 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-certs\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686498 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686432 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-data\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686760 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686717 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-data\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.686913 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.686889 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5029fa33-453d-46cf-b6b3-44ea0568bad2-istiod-ca-cert\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.688324 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.688299 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-envoy\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.688678 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.688659 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-podinfo\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.697944 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.697909 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjkxz\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-kube-api-access-mjkxz\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.699355 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.699317 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-token\") pod \"e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:24.799573 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.799533 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:24.867897 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:24.867794 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:25.048634 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.048592 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g"] Jul 10 23:15:25.050370 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:15:25.050335 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5029fa33_453d_46cf_b6b3_44ea0568bad2.slice/crio-d38bb17dc85b79791020a5dbe68ec32f46ce4cb4a347724f425069076fe50bfd WatchSource:0}: Error finding container d38bb17dc85b79791020a5dbe68ec32f46ce4cb4a347724f425069076fe50bfd: Status 404 returned error can't find the container with id d38bb17dc85b79791020a5dbe68ec32f46ce4cb4a347724f425069076fe50bfd Jul 10 23:15:25.052651 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.052585 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:25.052701 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.052688 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:25.052744 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.052715 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:25.360501 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.360459 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" event={"ID":"5029fa33-453d-46cf-b6b3-44ea0568bad2","Type":"ContainerStarted","Data":"77302239b5073b311a89131eed48fe126eb1453c00b63a17ec99b6acfb4166a2"} Jul 10 23:15:25.360704 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.360528 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" event={"ID":"5029fa33-453d-46cf-b6b3-44ea0568bad2","Type":"ContainerStarted","Data":"d38bb17dc85b79791020a5dbe68ec32f46ce4cb4a347724f425069076fe50bfd"} Jul 10 23:15:25.381881 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.381825 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podStartSLOduration=1.381807907 podStartE2EDuration="1.381807907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:15:25.380677064 +0000 UTC m=+1984.018102738" watchObservedRunningTime="2026-07-10 23:15:25.381807907 +0000 UTC m=+1984.019233581" Jul 10 23:15:25.868599 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:25.868555 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:26.874627 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:26.874593 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:27.288103 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.288038 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:27.373684 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.373644 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:27.375154 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.375128 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:27.386650 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.386613 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:27.486882 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.486848 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:27.614759 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.614651 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:27.804423 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.804382 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:15:27.943478 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:27.943384 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:28.048510 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:28.048458 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:31.779448 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:31.779385 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:34.745601 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:34.745562 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:35.361469 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:35.361416 2551 patch_prober.go:28] interesting pod/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 23:15:35.361655 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:35.361506 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 23:15:36.429821 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:36.429735 2551 generic.go:356] "Generic (PLEG): container finished" podID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerID="9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080" exitCode=0 Jul 10 23:15:36.430290 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:36.429808 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" event={"ID":"c7f7c5dd-0810-4b1d-b745-aacde8005e45","Type":"ContainerDied","Data":"9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080"} Jul 10 23:15:36.430290 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:36.430260 2551 scope.go:117] "RemoveContainer" containerID="9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080" Jul 10 23:15:36.431379 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:36.431351 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:36.431444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:36.431421 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:36.431481 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:36.431459 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:37.437324 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:37.437282 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" event={"ID":"c7f7c5dd-0810-4b1d-b745-aacde8005e45","Type":"ContainerStarted","Data":"cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc"} Jul 10 23:15:37.940646 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:37.940595 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:37.940984 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:37.940953 2551 patch_prober.go:28] interesting pod/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 23:15:37.941113 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:37.941010 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 23:15:38.940837 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:38.940795 2551 patch_prober.go:28] interesting pod/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 23:15:38.941313 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:38.940858 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 23:15:39.578949 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:39.578912 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:39.679002 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:39.678962 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:39.828821 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:39.828782 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:39.919175 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:39.918950 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:39.940950 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:39.940915 2551 patch_prober.go:28] interesting pod/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 23:15:39.941399 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:39.940973 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 23:15:40.205640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:40.205546 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:40.940974 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:40.940936 2551 patch_prober.go:28] interesting pod/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 23:15:40.941400 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:40.941009 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 23:15:41.944016 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:41.943980 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:41.944447 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:41.944377 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:41.945176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:41.945151 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:43.000922 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:43.000885 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:15:43.415436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:43.415382 2551 patch_prober.go:28] interesting pod/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 23:15:43.415627 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:43.415470 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 23:15:44.129261 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.129205 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:44.478533 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.478442 2551 generic.go:356] "Generic (PLEG): container finished" podID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerID="77302239b5073b311a89131eed48fe126eb1453c00b63a17ec99b6acfb4166a2" exitCode=0 Jul 10 23:15:44.478533 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.478480 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" event={"ID":"5029fa33-453d-46cf-b6b3-44ea0568bad2","Type":"ContainerDied","Data":"77302239b5073b311a89131eed48fe126eb1453c00b63a17ec99b6acfb4166a2"} Jul 10 23:15:44.478891 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.478875 2551 scope.go:117] "RemoveContainer" containerID="77302239b5073b311a89131eed48fe126eb1453c00b63a17ec99b6acfb4166a2" Jul 10 23:15:44.479910 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.479881 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:44.480007 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.479950 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:44.480007 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:44.479979 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:45.489905 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:45.489844 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" event={"ID":"5029fa33-453d-46cf-b6b3-44ea0568bad2","Type":"ContainerStarted","Data":"9c37ef68a12d44b86edd27bce1ed03bc39ce68ffe83348cd595c118a31fdc45b"} Jul 10 23:15:45.868598 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:45.868497 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:45.868598 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:45.868590 2551 patch_prober.go:28] interesting pod/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 23:15:45.868903 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:45.868658 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 23:15:46.372881 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.372836 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:46.473029 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.472988 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:46.608771 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.608734 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs"] Jul 10 23:15:46.609176 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.608960 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" containerID="cri-o://cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc" gracePeriod=30 Jul 10 23:15:46.637384 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.637287 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:46.849021 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.848986 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:46.869364 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.869315 2551 patch_prober.go:28] interesting pod/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 23:15:46.870271 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:46.869720 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 23:15:47.054150 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:47.054111 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:47.444379 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:47.444279 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:47.677409 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:47.677366 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:47.787550 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:47.787516 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:47.868600 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:47.868551 2551 patch_prober.go:28] interesting pod/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 23:15:47.868834 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:47.868636 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 23:15:48.098398 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:48.098305 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:48.868631 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:48.868571 2551 patch_prober.go:28] interesting pod/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 23:15:48.869167 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:48.868661 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 23:15:49.872251 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:49.872187 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:49.872661 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:49.872497 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:49.873351 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:49.873335 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:51.880914 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.880844 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:51.959440 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959403 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-credential-socket\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959440 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959455 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-certs\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959670 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959490 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-podinfo\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959670 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959515 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istiod-ca-cert\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959670 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959556 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-socket\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959670 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959581 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chhs2\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-kube-api-access-chhs2\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959761 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-data\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.959875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959797 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:51.959875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959812 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:51.959875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959818 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-token\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.960091 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.959876 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-envoy\") pod \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\" (UID: \"c7f7c5dd-0810-4b1d-b745-aacde8005e45\") " Jul 10 23:15:51.960091 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960053 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:51.960091 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960068 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:15:51.960330 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960308 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:51.960407 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960335 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:51.960407 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960349 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:51.960407 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960364 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:51.960689 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.960663 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:51.961966 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.961935 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:15:51.962077 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.961992 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:51.962077 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.962025 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-kube-api-access-chhs2" (OuterVolumeSpecName: "kube-api-access-chhs2") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "kube-api-access-chhs2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:51.962077 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:51.962040 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7f7c5dd-0810-4b1d-b745-aacde8005e45" (UID: "c7f7c5dd-0810-4b1d-b745-aacde8005e45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:52.061170 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.061133 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chhs2\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-kube-api-access-chhs2\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:52.061170 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.061167 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:52.061170 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.061179 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:52.061424 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.061191 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:52.061424 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.061202 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7f7c5dd-0810-4b1d-b745-aacde8005e45-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:52.531948 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.531905 2551 generic.go:356] "Generic (PLEG): container finished" podID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerID="cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc" exitCode=0 Jul 10 23:15:52.532139 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.531956 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" event={"ID":"c7f7c5dd-0810-4b1d-b745-aacde8005e45","Type":"ContainerDied","Data":"cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc"} Jul 10 23:15:52.532139 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.532006 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" event={"ID":"c7f7c5dd-0810-4b1d-b745-aacde8005e45","Type":"ContainerDied","Data":"ffbee129b31775903d3c37988087ea16c18f86fdeb0edb40da5499796d49ba76"} Jul 10 23:15:52.532139 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.532029 2551 scope.go:117] "RemoveContainer" containerID="cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc" Jul 10 23:15:52.532139 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.531960 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs" Jul 10 23:15:52.541474 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.541453 2551 scope.go:117] "RemoveContainer" containerID="9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080" Jul 10 23:15:52.549493 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.549462 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs"] Jul 10 23:15:52.550927 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.550906 2551 scope.go:117] "RemoveContainer" containerID="cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc" Jul 10 23:15:52.551511 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:15:52.551482 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc\": container with ID starting with cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc not found: ID does not exist" containerID="cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc" Jul 10 23:15:52.551774 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.551520 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc"} err="failed to get container status \"cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc\": rpc error: code = NotFound desc = could not find container \"cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc\": container with ID starting with cc24cf67ebbf91b4c9cff98b24b796cb05b6c2f5cbe283fc124424d7e7934dfc not found: ID does not exist" Jul 10 23:15:52.551774 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.551548 2551 scope.go:117] "RemoveContainer" containerID="9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080" Jul 10 23:15:52.552009 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:15:52.551981 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080\": container with ID starting with 9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080 not found: ID does not exist" containerID="9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080" Jul 10 23:15:52.552120 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.552019 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080"} err="failed to get container status \"9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080\": rpc error: code = NotFound desc = could not find container \"9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080\": container with ID starting with 9df56ea990796431576509349a94da25cf7ec8c7878e8cdef384412ab388a080 not found: ID does not exist" Jul 10 23:15:52.554165 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:52.554142 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d639f4-openshift-default-78b6dcf8c9-wpcvs"] Jul 10 23:15:53.042809 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.042775 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:53.171895 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.171861 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:53.307433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.307320 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g"] Jul 10 23:15:53.307617 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.307553 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" containerID="cri-o://9c37ef68a12d44b86edd27bce1ed03bc39ce68ffe83348cd595c118a31fdc45b" gracePeriod=30 Jul 10 23:15:53.329953 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.329917 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:53.547380 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.547345 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:53.873607 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:53.873567 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:54.041964 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.041924 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" path="/var/lib/kubelet/pods/c7f7c5dd-0810-4b1d-b745-aacde8005e45/volumes" Jul 10 23:15:54.152687 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.152576 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn"] Jul 10 23:15:54.153241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153199 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" Jul 10 23:15:54.153241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153236 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" Jul 10 23:15:54.153398 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153262 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" Jul 10 23:15:54.153398 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153271 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e03011b-e0bf-4c6e-ae76-d602ae68e256" containerName="istio-proxy" Jul 10 23:15:54.153398 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153303 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" Jul 10 23:15:54.153398 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153315 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" Jul 10 23:15:54.153597 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153403 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" Jul 10 23:15:54.153597 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.153419 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f7c5dd-0810-4b1d-b745-aacde8005e45" containerName="istio-proxy" Jul 10 23:15:54.158341 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.158314 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.161938 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.161908 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-6d6384-openshift-default-dockercfg-phvz4\"" Jul 10 23:15:54.172013 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.171979 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn"] Jul 10 23:15:54.178065 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.178035 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:54.285166 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285120 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-certs\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285178 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-credential-socket\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285197 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-token\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285324 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-podinfo\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285372 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-envoy\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285399 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-data\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285436 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1996e2c6-ffea-46bf-9318-416c574d6c16-istiod-ca-cert\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285485 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5lxr\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-kube-api-access-f5lxr\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.285640 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.285523 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-socket\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386351 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386308 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-certs\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386370 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-credential-socket\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386487 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-token\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386658 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386565 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-podinfo\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386658 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386601 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-envoy\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386658 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386629 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-data\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386676 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1996e2c6-ffea-46bf-9318-416c574d6c16-istiod-ca-cert\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386805 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386790 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-credential-socket\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386922 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386721 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5lxr\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-kube-api-access-f5lxr\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386922 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386827 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-certs\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.386922 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.386846 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-socket\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.387074 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.387032 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-data\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.387135 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.387121 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-socket\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.387433 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.387408 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1996e2c6-ffea-46bf-9318-416c574d6c16-istiod-ca-cert\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.389121 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.389098 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-envoy\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.389387 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.389370 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-podinfo\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.395094 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.395059 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-token\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.395305 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.395283 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5lxr\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-kube-api-access-f5lxr\") pod \"e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.469954 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.469859 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:54.651846 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.651816 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn"] Jul 10 23:15:54.654464 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:15:54.654431 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1996e2c6_ffea_46bf_9318_416c574d6c16.slice/crio-8ac6afb9767bb7897ed4aa65e3c7b79b2dcea9ca8ed94ec56f47b194af5725af WatchSource:0}: Error finding container 8ac6afb9767bb7897ed4aa65e3c7b79b2dcea9ca8ed94ec56f47b194af5725af: Status 404 returned error can't find the container with id 8ac6afb9767bb7897ed4aa65e3c7b79b2dcea9ca8ed94ec56f47b194af5725af Jul 10 23:15:54.656743 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.656712 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:54.656864 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.656792 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:54.656864 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.656839 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:15:54.762490 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:54.762452 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:55.553079 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:55.553046 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" event={"ID":"1996e2c6-ffea-46bf-9318-416c574d6c16","Type":"ContainerStarted","Data":"553207a9c2b05fe5863e6238fb38dcee90d2affb0ca90bf9d1814a8a92d2b96e"} Jul 10 23:15:55.553522 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:55.553100 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" event={"ID":"1996e2c6-ffea-46bf-9318-416c574d6c16","Type":"ContainerStarted","Data":"8ac6afb9767bb7897ed4aa65e3c7b79b2dcea9ca8ed94ec56f47b194af5725af"} Jul 10 23:15:55.577950 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:55.577863 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podStartSLOduration=1.5778391840000001 podStartE2EDuration="1.577839184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:15:55.574975955 +0000 UTC m=+2014.212401631" watchObservedRunningTime="2026-07-10 23:15:55.577839184 +0000 UTC m=+2014.215264850" Jul 10 23:15:56.470741 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:56.470702 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:56.476187 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:56.476158 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:56.558739 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:56.558708 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:56.559831 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:56.559808 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:15:57.478693 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:57.478659 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:57.550671 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:57.550641 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:57.625686 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:57.625656 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:57.783104 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:57.783073 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:58.044387 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.044299 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:58.147323 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.147288 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:15:58.573194 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.573099 2551 generic.go:356] "Generic (PLEG): container finished" podID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerID="9c37ef68a12d44b86edd27bce1ed03bc39ce68ffe83348cd595c118a31fdc45b" exitCode=0 Jul 10 23:15:58.573194 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.573171 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" event={"ID":"5029fa33-453d-46cf-b6b3-44ea0568bad2","Type":"ContainerDied","Data":"9c37ef68a12d44b86edd27bce1ed03bc39ce68ffe83348cd595c118a31fdc45b"} Jul 10 23:15:58.573426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.573280 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" event={"ID":"5029fa33-453d-46cf-b6b3-44ea0568bad2","Type":"ContainerDied","Data":"d38bb17dc85b79791020a5dbe68ec32f46ce4cb4a347724f425069076fe50bfd"} Jul 10 23:15:58.573426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.573302 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d38bb17dc85b79791020a5dbe68ec32f46ce4cb4a347724f425069076fe50bfd" Jul 10 23:15:58.573426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.573318 2551 scope.go:117] "RemoveContainer" containerID="77302239b5073b311a89131eed48fe126eb1453c00b63a17ec99b6acfb4166a2" Jul 10 23:15:58.587912 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.587886 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:58.728719 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728663 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-data\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728749 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjkxz\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-kube-api-access-mjkxz\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728786 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-certs\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728829 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-socket\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728901 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-credential-socket\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728932 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-envoy\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.728964 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-token\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729000 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5029fa33-453d-46cf-b6b3-44ea0568bad2-istiod-ca-cert\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729038 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-podinfo\") pod \"5029fa33-453d-46cf-b6b3-44ea0568bad2\" (UID: \"5029fa33-453d-46cf-b6b3-44ea0568bad2\") " Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729116 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729132 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:58.729241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729152 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:58.729853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729488 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5029fa33-453d-46cf-b6b3-44ea0568bad2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:15:58.729853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729581 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.729853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729599 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.729853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729612 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5029fa33-453d-46cf-b6b3-44ea0568bad2-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.729853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729626 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.729853 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.729680 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-data" (OuterVolumeSpecName: "istio-data") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:58.731341 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.731316 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:15:58.731447 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.731345 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:15:58.731503 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.731482 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-kube-api-access-mjkxz" (OuterVolumeSpecName: "kube-api-access-mjkxz") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "kube-api-access-mjkxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:58.731799 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.731782 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-token" (OuterVolumeSpecName: "istio-token") pod "5029fa33-453d-46cf-b6b3-44ea0568bad2" (UID: "5029fa33-453d-46cf-b6b3-44ea0568bad2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:15:58.817313 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.817274 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:15:58.830183 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.830083 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjkxz\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-kube-api-access-mjkxz\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.830183 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.830118 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.830183 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.830132 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.830183 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.830145 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:58.830183 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:58.830156 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5029fa33-453d-46cf-b6b3-44ea0568bad2-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:15:59.579952 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:59.579921 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g" Jul 10 23:15:59.614850 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:59.614809 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g"] Jul 10 23:15:59.618867 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:15:59.618833 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8b0687-openshift-default-789c459d95-rcj7g"] Jul 10 23:16:00.042872 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:00.042833 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" path="/var/lib/kubelet/pods/5029fa33-453d-46cf-b6b3-44ea0568bad2/volumes" Jul 10 23:16:01.961200 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:01.961153 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:02.303589 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.303552 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:02.687416 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.687327 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj"] Jul 10 23:16:02.687744 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.687731 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" Jul 10 23:16:02.687800 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.687745 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" Jul 10 23:16:02.687839 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.687825 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" Jul 10 23:16:02.687839 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.687833 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" Jul 10 23:16:02.692467 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.692438 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.696073 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.696046 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-02c498-openshift-default-dockercfg-hgzbm\"" Jul 10 23:16:02.712937 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.712900 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj"] Jul 10 23:16:02.754030 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.753988 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:02.764192 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764145 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-podinfo\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764192 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764191 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589d34ed-13e0-4b6d-b44c-7448575c80fe-istiod-ca-cert\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764495 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764242 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-credential-socket\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764495 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764372 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-token\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764495 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764466 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-data\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764668 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764505 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frtzr\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-kube-api-access-frtzr\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764668 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764528 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-envoy\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764668 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764548 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-certs\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.764668 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.764577 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-socket\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.844803 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.844771 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:02.865092 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865058 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frtzr\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-kube-api-access-frtzr\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.865349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865099 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-envoy\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.865349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865126 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-certs\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.865349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865150 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-socket\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.865349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865196 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-podinfo\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.865349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865257 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589d34ed-13e0-4b6d-b44c-7448575c80fe-istiod-ca-cert\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.865349 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865294 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-credential-socket\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866178 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865751 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-token\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866178 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865852 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-data\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866178 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.865940 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-certs\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866178 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.866152 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589d34ed-13e0-4b6d-b44c-7448575c80fe-istiod-ca-cert\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866477 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.866190 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-data\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866686 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.866649 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-socket\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.866967 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.866938 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-credential-socket\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.869540 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.869513 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-podinfo\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.869683 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.869515 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-envoy\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.873811 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.873784 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-token\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:02.873811 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:02.873806 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frtzr\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-kube-api-access-frtzr\") pod \"e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:03.005034 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.004999 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:03.162440 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.162411 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj"] Jul 10 23:16:03.165051 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:16:03.165020 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589d34ed_13e0_4b6d_b44c_7448575c80fe.slice/crio-543e61662e8fb221c4fab7009d8c1a5d98870da8786448376b96827522a4b940 WatchSource:0}: Error finding container 543e61662e8fb221c4fab7009d8c1a5d98870da8786448376b96827522a4b940: Status 404 returned error can't find the container with id 543e61662e8fb221c4fab7009d8c1a5d98870da8786448376b96827522a4b940 Jul 10 23:16:03.167488 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.167452 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:03.167597 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.167524 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:03.167597 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.167553 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:03.603423 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.603378 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" event={"ID":"589d34ed-13e0-4b6d-b44c-7448575c80fe","Type":"ContainerStarted","Data":"2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e"} Jul 10 23:16:03.603616 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.603471 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" event={"ID":"589d34ed-13e0-4b6d-b44c-7448575c80fe","Type":"ContainerStarted","Data":"543e61662e8fb221c4fab7009d8c1a5d98870da8786448376b96827522a4b940"} Jul 10 23:16:03.623550 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:03.623491 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podStartSLOduration=1.623473673 podStartE2EDuration="1.623473673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:16:03.62147268 +0000 UTC m=+2022.258898352" watchObservedRunningTime="2026-07-10 23:16:03.623473673 +0000 UTC m=+2022.260899347" Jul 10 23:16:04.006081 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:04.006042 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:04.011545 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:04.011512 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:04.609104 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:04.609066 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:04.610011 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:04.609989 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:05.508379 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:05.508341 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:05.596933 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:05.596892 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:05.673928 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:05.673877 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:06.201872 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:06.201837 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:06.303875 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:06.303834 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:10.047222 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:10.047177 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:12.498811 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:12.498767 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:16:12.601386 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:12.601343 2551 patch_prober.go:28] interesting pod/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 23:16:12.601635 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:12.601418 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 23:16:14.665956 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:14.665904 2551 generic.go:356] "Generic (PLEG): container finished" podID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerID="553207a9c2b05fe5863e6238fb38dcee90d2affb0ca90bf9d1814a8a92d2b96e" exitCode=0 Jul 10 23:16:14.666587 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:14.665978 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" event={"ID":"1996e2c6-ffea-46bf-9318-416c574d6c16","Type":"ContainerDied","Data":"553207a9c2b05fe5863e6238fb38dcee90d2affb0ca90bf9d1814a8a92d2b96e"} Jul 10 23:16:14.666587 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:14.666565 2551 scope.go:117] "RemoveContainer" containerID="553207a9c2b05fe5863e6238fb38dcee90d2affb0ca90bf9d1814a8a92d2b96e" Jul 10 23:16:14.668314 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:14.668270 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:14.668499 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:14.668376 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:14.668499 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:14.668424 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:15.677890 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:15.677825 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" event={"ID":"1996e2c6-ffea-46bf-9318-416c574d6c16","Type":"ContainerStarted","Data":"500cf210a6f7e0c447320cf6fdb9371713e7b6bb66aca9d96802114af722cf35"} Jul 10 23:16:16.471084 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:16.471041 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:16:16.471375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:16.471342 2551 patch_prober.go:28] interesting pod/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 23:16:16.471486 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:16.471426 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 23:16:17.470576 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:17.470534 2551 patch_prober.go:28] interesting pod/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 23:16:17.471232 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:17.471174 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 23:16:18.471721 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:18.471236 2551 patch_prober.go:28] interesting pod/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 23:16:18.471721 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:18.471308 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 23:16:19.471137 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:19.470673 2551 patch_prober.go:28] interesting pod/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 23:16:19.471137 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:19.470746 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 23:16:19.874103 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:19.874066 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:19.974395 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:19.974348 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:20.474652 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:20.474615 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:16:20.475151 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:20.475137 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:16:20.476041 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:20.476017 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:16:20.651464 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:20.651415 2551 patch_prober.go:28] interesting pod/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 23:16:20.651658 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:20.651497 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 23:16:21.720443 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:21.720409 2551 generic.go:356] "Generic (PLEG): container finished" podID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerID="2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e" exitCode=0 Jul 10 23:16:21.720898 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:21.720483 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" event={"ID":"589d34ed-13e0-4b6d-b44c-7448575c80fe","Type":"ContainerDied","Data":"2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e"} Jul 10 23:16:21.721051 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:21.721033 2551 scope.go:117] "RemoveContainer" containerID="2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e" Jul 10 23:16:21.722073 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:21.722046 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:21.722180 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:21.722117 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:21.722180 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:21.722151 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 23:16:22.727568 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:22.727528 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" event={"ID":"589d34ed-13e0-4b6d-b44c-7448575c80fe","Type":"ContainerStarted","Data":"3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920"} Jul 10 23:16:23.005608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:23.005567 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:23.005825 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:23.005798 2551 patch_prober.go:28] interesting pod/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 10 23:16:23.005878 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:23.005852 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 10 23:16:24.005750 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:24.005702 2551 patch_prober.go:28] interesting pod/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 10 23:16:24.006294 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:24.005772 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 10 23:16:25.005810 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:25.005771 2551 patch_prober.go:28] interesting pod/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 10 23:16:25.006272 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:25.005840 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 10 23:16:26.005931 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:26.005888 2551 patch_prober.go:28] interesting pod/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 10 23:16:26.006415 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:26.005949 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 10 23:16:26.955865 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:26.955825 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:27.009056 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:27.009025 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:27.009570 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:27.009510 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:27.010398 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:27.010379 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:16:27.034802 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:27.034769 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:27.678106 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:27.678070 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:31.497909 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:31.497867 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:16:50.659455 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:50.659356 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:16:54.496998 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:16:54.496959 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:17:08.929840 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:08.929805 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:17:14.302811 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:14.302769 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:17:15.093421 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:15.093387 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:17:22.139817 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:22.139788 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:17:22.141774 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:22.141741 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:17:30.797694 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:30.797656 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:17:36.412941 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:36.412907 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:17:47.801047 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:17:47.801006 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:18:04.398373 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:18:04.398328 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:18:23.694570 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:18:23.694533 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:18:40.809823 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:18:40.809783 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:18:56.760106 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:18:56.760058 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:18:57.745941 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:18:57.745906 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:18:59.399241 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:18:59.399185 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:19:06.927928 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:19:06.927889 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:19:21.391294 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:19:21.391244 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:19:28.055396 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:19:28.055363 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:19:33.890819 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:19:33.890776 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:19:48.302475 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:19:48.302368 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:20:04.786746 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:04.786709 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:20:05.732199 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:05.732164 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:20:08.160255 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:08.160194 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:20:08.497289 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:08.497161 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:20:24.592325 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:24.592281 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:20:40.034133 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:40.034090 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:20:40.327143 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:40.327046 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:20:40.391996 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:40.391954 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:20:44.489492 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:44.489439 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:20:59.293148 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:20:59.293107 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:21:16.195508 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:16.195467 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:21:20.607327 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:20.607291 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:20.711249 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:20.711181 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:23.991244 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:23.991191 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:24.826988 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:24.826953 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:27.779843 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:27.779805 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:27.881068 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:27.881033 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:30.702367 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:30.702327 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:21:44.607151 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:44.607111 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:45.166093 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:45.166044 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:45.309325 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:45.309288 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:45.412023 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:45.411988 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:45.580932 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:45.580899 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:49.389283 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:49.389239 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:21:51.117973 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.117943 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:51.190724 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.190690 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:51.364754 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.364714 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn"] Jul 10 23:21:51.365015 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.364981 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" containerID="cri-o://500cf210a6f7e0c447320cf6fdb9371713e7b6bb66aca9d96802114af722cf35" gracePeriod=30 Jul 10 23:21:51.387369 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.387277 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:51.603503 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.603465 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:51.755509 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:51.755469 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:52.147332 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:52.147244 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:52.365403 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:52.365366 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:52.475608 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:52.475516 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:52.634848 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:52.634808 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:56.525480 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.525445 2551 generic.go:356] "Generic (PLEG): container finished" podID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerID="500cf210a6f7e0c447320cf6fdb9371713e7b6bb66aca9d96802114af722cf35" exitCode=0 Jul 10 23:21:56.525908 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.525521 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" event={"ID":"1996e2c6-ffea-46bf-9318-416c574d6c16","Type":"ContainerDied","Data":"500cf210a6f7e0c447320cf6fdb9371713e7b6bb66aca9d96802114af722cf35"} Jul 10 23:21:56.525908 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.525591 2551 scope.go:117] "RemoveContainer" containerID="553207a9c2b05fe5863e6238fb38dcee90d2affb0ca90bf9d1814a8a92d2b96e" Jul 10 23:21:56.646410 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.646379 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:21:56.765050 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765013 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-certs\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765050 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765056 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-podinfo\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765102 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1996e2c6-ffea-46bf-9318-416c574d6c16-istiod-ca-cert\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765127 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-credential-socket\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765281 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-data\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765375 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765341 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-token\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765595 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765388 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:21:56.765595 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765407 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-envoy\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765595 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765461 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5lxr\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-kube-api-access-f5lxr\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765595 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765481 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-socket\") pod \"1996e2c6-ffea-46bf-9318-416c574d6c16\" (UID: \"1996e2c6-ffea-46bf-9318-416c574d6c16\") " Jul 10 23:21:56.765595 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765483 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:21:56.765863 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765612 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1996e2c6-ffea-46bf-9318-416c574d6c16-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:21:56.765863 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765827 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:21:56.766031 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765913 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.766031 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765934 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.766031 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765947 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1996e2c6-ffea-46bf-9318-416c574d6c16-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.766031 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.765960 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.766426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.766397 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-data" (OuterVolumeSpecName: "istio-data") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:21:56.767641 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.767610 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-kube-api-access-f5lxr" (OuterVolumeSpecName: "kube-api-access-f5lxr") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "kube-api-access-f5lxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:21:56.767753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.767661 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:21:56.767753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.767658 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-token" (OuterVolumeSpecName: "istio-token") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:21:56.767753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.767658 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1996e2c6-ffea-46bf-9318-416c574d6c16" (UID: "1996e2c6-ffea-46bf-9318-416c574d6c16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:21:56.866585 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.866539 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.866585 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.866578 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.866585 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.866587 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.866585 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.866595 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5lxr\" (UniqueName: \"kubernetes.io/projected/1996e2c6-ffea-46bf-9318-416c574d6c16-kube-api-access-f5lxr\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:56.866867 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:56.866605 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1996e2c6-ffea-46bf-9318-416c574d6c16-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:21:57.532426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:57.532396 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" Jul 10 23:21:57.532883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:57.532395 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn" event={"ID":"1996e2c6-ffea-46bf-9318-416c574d6c16","Type":"ContainerDied","Data":"8ac6afb9767bb7897ed4aa65e3c7b79b2dcea9ca8ed94ec56f47b194af5725af"} Jul 10 23:21:57.532883 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:57.532543 2551 scope.go:117] "RemoveContainer" containerID="500cf210a6f7e0c447320cf6fdb9371713e7b6bb66aca9d96802114af722cf35" Jul 10 23:21:57.555599 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:57.555561 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn"] Jul 10 23:21:57.558835 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:57.558795 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-6d6384-openshift-default-5589c459cf-7dpqn"] Jul 10 23:21:58.042413 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.042374 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" path="/var/lib/kubelet/pods/1996e2c6-ffea-46bf-9318-416c574d6c16/volumes" Jul 10 23:21:58.362194 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.362109 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:58.429967 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.429926 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:58.608003 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.607963 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj"] Jul 10 23:21:58.608515 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.608275 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" containerID="cri-o://3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920" gracePeriod=30 Jul 10 23:21:58.622898 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.622804 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:58.847684 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:58.847646 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:21:59.266699 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:21:59.266658 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55b7458fdb-btdhh_27167415-6a5d-4c1b-a196-c74e9ab0a0ad/authorino/0.log" Jul 10 23:22:02.495086 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:02.495050 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-kpxpw"] Jul 10 23:22:03.876296 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:03.876206 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:22:03.956131 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:03.956100 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-b2mll_6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c/manager/0.log" Jul 10 23:22:04.041165 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041135 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-envoy\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041165 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041169 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-credential-socket\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041186 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-data\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041225 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-token\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041264 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-certs\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041298 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-podinfo\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041330 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frtzr\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-kube-api-access-frtzr\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041707 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041562 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589d34ed-13e0-4b6d-b44c-7448575c80fe-istiod-ca-cert\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.041707 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041663 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-socket\") pod \"589d34ed-13e0-4b6d-b44c-7448575c80fe\" (UID: \"589d34ed-13e0-4b6d-b44c-7448575c80fe\") " Jul 10 23:22:04.042018 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.041989 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/589d34ed-13e0-4b6d-b44c-7448575c80fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 23:22:04.042098 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.042081 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589d34ed-13e0-4b6d-b44c-7448575c80fe-istiod-ca-cert\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.042189 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.042145 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:22:04.042189 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.042165 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:22:04.042353 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.042330 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:22:04.042585 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.042555 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:22:04.043673 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.043645 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 23:22:04.043787 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.043698 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 23:22:04.043974 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.043950 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-kube-api-access-frtzr" (OuterVolumeSpecName: "kube-api-access-frtzr") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "kube-api-access-frtzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:22:04.044374 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.044353 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "589d34ed-13e0-4b6d-b44c-7448575c80fe" (UID: "589d34ed-13e0-4b6d-b44c-7448575c80fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 23:22:04.074528 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.074495 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ks4l8_3d4a83cb-1fbf-4852-8cc2-9a91972f4186/manager/0.log" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143556 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-envoy\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143587 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-credential-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143600 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-data\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143610 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-token\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143620 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-certs\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143628 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589d34ed-13e0-4b6d-b44c-7448575c80fe-istio-podinfo\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143635 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frtzr\" (UniqueName: \"kubernetes.io/projected/589d34ed-13e0-4b6d-b44c-7448575c80fe-kube-api-access-frtzr\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.143653 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.143644 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589d34ed-13e0-4b6d-b44c-7448575c80fe-workload-socket\") on node \"ip-10-0-138-188.ec2.internal\" DevicePath \"\"" Jul 10 23:22:04.322331 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.322293 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-2hp5d_5544f4b6-217a-4264-9ce4-98a39eb48b3f/server/0.log" Jul 10 23:22:04.439675 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.439586 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hb647_3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38/manager/0.log" Jul 10 23:22:04.572718 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.572679 2551 generic.go:356] "Generic (PLEG): container finished" podID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerID="3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920" exitCode=0 Jul 10 23:22:04.572926 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.572743 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" Jul 10 23:22:04.572926 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.572757 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" event={"ID":"589d34ed-13e0-4b6d-b44c-7448575c80fe","Type":"ContainerDied","Data":"3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920"} Jul 10 23:22:04.572926 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.572851 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj" event={"ID":"589d34ed-13e0-4b6d-b44c-7448575c80fe","Type":"ContainerDied","Data":"543e61662e8fb221c4fab7009d8c1a5d98870da8786448376b96827522a4b940"} Jul 10 23:22:04.572926 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.572881 2551 scope.go:117] "RemoveContainer" containerID="3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920" Jul 10 23:22:04.583032 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.583012 2551 scope.go:117] "RemoveContainer" containerID="2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e" Jul 10 23:22:04.593160 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.593121 2551 scope.go:117] "RemoveContainer" containerID="3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920" Jul 10 23:22:04.593617 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:22:04.593588 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920\": container with ID starting with 3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920 not found: ID does not exist" containerID="3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920" Jul 10 23:22:04.593772 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.593742 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920"} err="failed to get container status \"3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920\": rpc error: code = NotFound desc = could not find container \"3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920\": container with ID starting with 3d9a3bdb7f080b7cf20abaa7787752e39cd88acfc97a52b528f25d82073e5920 not found: ID does not exist" Jul 10 23:22:04.593862 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.593777 2551 scope.go:117] "RemoveContainer" containerID="2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e" Jul 10 23:22:04.594181 ip-10-0-138-188 kubenswrapper[2551]: E0710 23:22:04.594148 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e\": container with ID starting with 2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e not found: ID does not exist" containerID="2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e" Jul 10 23:22:04.594278 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.594189 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e"} err="failed to get container status \"2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e\": rpc error: code = NotFound desc = could not find container \"2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e\": container with ID starting with 2fbc91af99dcceaeecb36ada05cd90ef38ba4537ec1337acdcf2d58a155c647e not found: ID does not exist" Jul 10 23:22:04.595569 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.595548 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj"] Jul 10 23:22:04.599463 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.599433 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-02c498-openshift-default-6f6cb49cf5-cvddj"] Jul 10 23:22:04.795645 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.795615 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845b6c684-vgpj5_59109aa9-8f44-4050-97e0-8828281f6916/copy-manifests/0.log" Jul 10 23:22:04.802524 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:04.802499 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845b6c684-vgpj5_59109aa9-8f44-4050-97e0-8828281f6916/manager/0.log" Jul 10 23:22:05.254250 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:05.254205 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6sbt2_e32f1d5d-f6fa-410f-ab26-b60faede8696/postgres/0.log" Jul 10 23:22:06.044150 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:06.044106 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" path="/var/lib/kubelet/pods/589d34ed-13e0-4b6d-b44c-7448575c80fe/volumes" Jul 10 23:22:06.493858 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:06.493758 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55b7458fdb-btdhh_27167415-6a5d-4c1b-a196-c74e9ab0a0ad/authorino/0.log" Jul 10 23:22:06.609614 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:06.609581 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nqcsc_073c116b-ef42-4331-a3f9-fdc7bea364fa/manager/0.log" Jul 10 23:22:06.939269 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:06.939238 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7cmph_866b30b1-0a41-4b39-addf-4900a909f1d3/registry-server/0.log" Jul 10 23:22:07.170646 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:07.170618 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-kpxpw_4c674571-05ec-49d0-990a-24bb1a026a69/limitador/0.log" Jul 10 23:22:07.622754 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:07.622720 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678p8qr6_4e903ff9-e362-40e6-838e-084fbf1c07d8/istio-proxy/0.log" Jul 10 23:22:07.967804 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:07.967709 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7454dcb6b8-kt2gx_3ca65be0-4a38-41a7-b4b8-f2f44dbe7556/kube-auth-proxy/0.log" Jul 10 23:22:08.081863 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:08.081820 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/1.log" Jul 10 23:22:08.195321 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:08.195284 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-k4msn_53349d7d-81ab-46cf-88b3-75b931a737bd/payload-pre-processing/0.log" Jul 10 23:22:08.305551 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:08.305520 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7zpj_a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1/payload-processing/0.log" Jul 10 23:22:08.412704 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:08.412669 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b57bc954d-h2spw_9e113a39-f272-431f-a6e5-818d5a373fc8/router/0.log" Jul 10 23:22:09.073614 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:09.073580 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbc6b45_2824666d-59d9-40d9-a3c1-a3d0b2e01491/main/0.log" Jul 10 23:22:12.839224 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:12.839174 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:22:12.912735 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:12.912695 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" limit=1024 Jul 10 23:22:15.707043 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:15.707014 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qgkfr_60803b6d-b5c5-4c66-b602-f4e7c229ce13/global-pull-secret-syncer/0.log" Jul 10 23:22:15.759045 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:15.759009 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l5rk7_2e19610a-10bf-4e53-8a48-eb5bb7363d1a/konnectivity-agent/0.log" Jul 10 23:22:15.888962 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:15.888933 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-188.ec2.internal_2a475e5bcf6b6d12915427e27f6213c9/haproxy/0.log" Jul 10 23:22:20.609518 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:20.609475 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55b7458fdb-btdhh_27167415-6a5d-4c1b-a196-c74e9ab0a0ad/authorino/0.log" Jul 10 23:22:20.643274 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:20.643200 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nqcsc_073c116b-ef42-4331-a3f9-fdc7bea364fa/manager/0.log" Jul 10 23:22:20.741753 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:20.741717 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7cmph_866b30b1-0a41-4b39-addf-4900a909f1d3/registry-server/0.log" Jul 10 23:22:20.972626 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:20.972531 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-kpxpw_4c674571-05ec-49d0-990a-24bb1a026a69/limitador/0.log" Jul 10 23:22:22.171834 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:22.171803 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:22:22.175328 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:22.175304 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:22:22.804488 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:22.804450 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-p5drw_90b8a8ae-1d63-4e73-8d39-2821e3ec325b/cluster-monitoring-operator/0.log" Jul 10 23:22:23.146068 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:23.145977 2551 scope.go:117] "RemoveContainer" containerID="9c37ef68a12d44b86edd27bce1ed03bc39ce68ffe83348cd595c118a31fdc45b" Jul 10 23:22:23.185417 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:23.185377 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xc4rg_dc13cbb9-e635-4295-9d31-79caba9dc5b6/node-exporter/0.log" Jul 10 23:22:23.212016 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:23.211987 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xc4rg_dc13cbb9-e635-4295-9d31-79caba9dc5b6/kube-rbac-proxy/0.log" Jul 10 23:22:23.249336 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:23.249306 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xc4rg_dc13cbb9-e635-4295-9d31-79caba9dc5b6/init-textfile/0.log" Jul 10 23:22:25.186854 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:25.186827 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-4lvqn_b235a06e-ca2f-406a-ac42-1810e20b6f7c/networking-console-plugin/0.log" Jul 10 23:22:25.824980 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:25.824946 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/2.log" Jul 10 23:22:25.829405 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:25.829378 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kk4fj_09c729d5-a628-44ee-aaff-979c8a614328/console-operator/3.log" Jul 10 23:22:26.439928 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.439894 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xz2px/perf-node-gather-daemonset-td542"] Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440319 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440331 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440340 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440346 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440354 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440366 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5029fa33-453d-46cf-b6b3-44ea0568bad2" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440375 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440383 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440392 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" Jul 10 23:22:26.440436 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440400 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" Jul 10 23:22:26.440955 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440494 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1996e2c6-ffea-46bf-9318-416c574d6c16" containerName="istio-proxy" Jul 10 23:22:26.440955 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440508 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" Jul 10 23:22:26.440955 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.440517 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="589d34ed-13e0-4b6d-b44c-7448575c80fe" containerName="istio-proxy" Jul 10 23:22:26.445599 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.445572 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.447989 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.447955 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xz2px\"/\"default-dockercfg-shbnr\"" Jul 10 23:22:26.448114 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.447956 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xz2px\"/\"kube-root-ca.crt\"" Jul 10 23:22:26.449041 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.449019 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xz2px\"/\"openshift-service-ca.crt\"" Jul 10 23:22:26.454529 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.454400 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xz2px/perf-node-gather-daemonset-td542"] Jul 10 23:22:26.542945 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.542898 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-proc\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.543141 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.543003 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-sys\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.543141 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.543030 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-lib-modules\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.543141 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.543046 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc5nc\" (UniqueName: \"kubernetes.io/projected/05d653df-8cff-47a6-a4c0-0ccaf46839bd-kube-api-access-xc5nc\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.543141 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.543070 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-podres\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644302 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644263 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-proc\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644460 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644330 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-sys\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644460 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644359 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-lib-modules\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644460 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644376 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc5nc\" (UniqueName: \"kubernetes.io/projected/05d653df-8cff-47a6-a4c0-0ccaf46839bd-kube-api-access-xc5nc\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644460 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644400 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-podres\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644460 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644400 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-proc\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644664 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644462 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-sys\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644664 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644537 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-lib-modules\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.644664 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.644551 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05d653df-8cff-47a6-a4c0-0ccaf46839bd-podres\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.652450 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.652414 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc5nc\" (UniqueName: \"kubernetes.io/projected/05d653df-8cff-47a6-a4c0-0ccaf46839bd-kube-api-access-xc5nc\") pod \"perf-node-gather-daemonset-td542\" (UID: \"05d653df-8cff-47a6-a4c0-0ccaf46839bd\") " pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.757716 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.757681 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:26.902235 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:26.902193 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xz2px/perf-node-gather-daemonset-td542"] Jul 10 23:22:26.904454 ip-10-0-138-188 kubenswrapper[2551]: W0710 23:22:26.904410 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod05d653df_8cff_47a6_a4c0_0ccaf46839bd.slice/crio-21e95dfcbb3e0561a4dbcb94a04cb0427519dba5a864d3bf42a0b9f75b0f63eb WatchSource:0}: Error finding container 21e95dfcbb3e0561a4dbcb94a04cb0427519dba5a864d3bf42a0b9f75b0f63eb: Status 404 returned error can't find the container with id 21e95dfcbb3e0561a4dbcb94a04cb0427519dba5a864d3bf42a0b9f75b0f63eb Jul 10 23:22:27.172276 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:27.172168 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-bdjkm_6e04e702-7813-4ab9-81f2-41d407e0afe8/volume-data-source-validator/0.log" Jul 10 23:22:27.694650 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:27.694611 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" event={"ID":"05d653df-8cff-47a6-a4c0-0ccaf46839bd","Type":"ContainerStarted","Data":"18d5d49a8dc6565cdab1ce8e93b19aa61233c17a033e8fde95c32939f57b0820"} Jul 10 23:22:27.695066 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:27.694668 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" event={"ID":"05d653df-8cff-47a6-a4c0-0ccaf46839bd","Type":"ContainerStarted","Data":"21e95dfcbb3e0561a4dbcb94a04cb0427519dba5a864d3bf42a0b9f75b0f63eb"} Jul 10 23:22:27.695066 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:27.694720 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:27.711960 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:27.711901 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" podStartSLOduration=1.711882885 podStartE2EDuration="1.711882885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 23:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:22:27.710042858 +0000 UTC m=+2406.347468538" watchObservedRunningTime="2026-07-10 23:22:27.711882885 +0000 UTC m=+2406.349308559" Jul 10 23:22:28.202000 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:28.201969 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4n7g4_354a05bd-82c0-4502-8e66-468ad5da070a/dns/0.log" Jul 10 23:22:28.272750 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:28.272724 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4n7g4_354a05bd-82c0-4502-8e66-468ad5da070a/kube-rbac-proxy/0.log" Jul 10 23:22:28.472948 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:28.472863 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wm5gq_098d0cd2-46a5-4c65-85cc-8a3ee65bf3a7/dns-node-resolver/0.log" Jul 10 23:22:29.099702 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:29.099676 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rx6mq_0f40b374-3519-464b-b29e-c3c2303f7fa8/node-ca/0.log" Jul 10 23:22:30.021967 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.021937 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678p8qr6_4e903ff9-e362-40e6-838e-084fbf1c07d8/istio-proxy/0.log" Jul 10 23:22:30.278271 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.278153 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7454dcb6b8-kt2gx_3ca65be0-4a38-41a7-b4b8-f2f44dbe7556/kube-auth-proxy/0.log" Jul 10 23:22:30.310151 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.310118 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/0.log" Jul 10 23:22:30.338438 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.338399 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/1.log" Jul 10 23:22:30.371663 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.371628 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-k4msn_53349d7d-81ab-46cf-88b3-75b931a737bd/payload-pre-processing/0.log" Jul 10 23:22:30.406937 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.406905 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7zpj_a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1/payload-processing/0.log" Jul 10 23:22:30.432507 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:30.432476 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b57bc954d-h2spw_9e113a39-f272-431f-a6e5-818d5a373fc8/router/0.log" Jul 10 23:22:31.157401 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:31.157374 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dj8lx_b76215f8-9338-4d83-a9e2-62779b4f100f/serve-healthcheck-canary/0.log" Jul 10 23:22:31.742545 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:31.742506 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-p7ldl_db6471ad-9a09-4ecc-8af9-8a0659846986/insights-operator/0.log" Jul 10 23:22:31.743222 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:31.743188 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-p7ldl_db6471ad-9a09-4ecc-8af9-8a0659846986/insights-operator/1.log" Jul 10 23:22:31.877415 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:31.877384 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzs4r_a6372247-d610-4b24-83e0-b220e42ca4ab/kube-rbac-proxy/0.log" Jul 10 23:22:31.903301 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:31.903273 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzs4r_a6372247-d610-4b24-83e0-b220e42ca4ab/exporter/0.log" Jul 10 23:22:31.928151 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:31.928123 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nzs4r_a6372247-d610-4b24-83e0-b220e42ca4ab/extractor/0.log" Jul 10 23:22:33.710266 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:33.710238 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xz2px/perf-node-gather-daemonset-td542" Jul 10 23:22:34.125672 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:34.125641 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-b2mll_6d0a9bc9-f645-4bbe-b8f0-c7e095a2d72c/manager/0.log" Jul 10 23:22:34.265426 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:34.265387 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ks4l8_3d4a83cb-1fbf-4852-8cc2-9a91972f4186/manager/0.log" Jul 10 23:22:34.440196 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:34.440103 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-2hp5d_5544f4b6-217a-4264-9ce4-98a39eb48b3f/server/0.log" Jul 10 23:22:34.506760 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:34.506729 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hb647_3cb8165b-ffd5-4b5c-9ee8-01e75ac11b38/manager/0.log" Jul 10 23:22:34.715372 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:34.715293 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845b6c684-vgpj5_59109aa9-8f44-4050-97e0-8828281f6916/manager/0.log" Jul 10 23:22:34.737268 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:34.737234 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845b6c684-vgpj5_59109aa9-8f44-4050-97e0-8828281f6916/copy-manifests/0.log" Jul 10 23:22:36.918101 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:36.918010 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678p8qr6_4e903ff9-e362-40e6-838e-084fbf1c07d8/istio-proxy/0.log" Jul 10 23:22:37.131435 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:37.131409 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7454dcb6b8-kt2gx_3ca65be0-4a38-41a7-b4b8-f2f44dbe7556/kube-auth-proxy/0.log" Jul 10 23:22:37.151895 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:37.151852 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/0.log" Jul 10 23:22:37.182667 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:37.182579 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/1.log" Jul 10 23:22:37.201694 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:37.201656 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-k4msn_53349d7d-81ab-46cf-88b3-75b931a737bd/payload-pre-processing/0.log" Jul 10 23:22:37.220786 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:37.220752 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7zpj_a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1/payload-processing/0.log" Jul 10 23:22:37.235445 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:37.235414 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b57bc954d-h2spw_9e113a39-f272-431f-a6e5-818d5a373fc8/router/0.log" Jul 10 23:22:38.474444 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.474356 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678p8qr6_4e903ff9-e362-40e6-838e-084fbf1c07d8/istio-proxy/0.log" Jul 10 23:22:38.685606 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.685567 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7454dcb6b8-kt2gx_3ca65be0-4a38-41a7-b4b8-f2f44dbe7556/kube-auth-proxy/0.log" Jul 10 23:22:38.710598 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.710571 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/0.log" Jul 10 23:22:38.742231 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.742118 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lpbbh_27ab5b4c-6e90-4d13-99f4-bff938e21213/istio-proxy/1.log" Jul 10 23:22:38.766128 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.766074 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-k4msn_53349d7d-81ab-46cf-88b3-75b931a737bd/payload-pre-processing/0.log" Jul 10 23:22:38.788648 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.788601 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7zpj_a7ab5dcf-953b-48eb-b6e5-ea03fda6fad1/payload-processing/0.log" Jul 10 23:22:38.808200 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:38.808172 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b57bc954d-h2spw_9e113a39-f272-431f-a6e5-818d5a373fc8/router/0.log" Jul 10 23:22:41.256929 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:41.256892 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5ng5s_734bd34c-bb9b-4254-9e81-71ccce05eec8/migrator/0.log" Jul 10 23:22:41.287046 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:41.287016 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5ng5s_734bd34c-bb9b-4254-9e81-71ccce05eec8/graceful-termination/0.log" Jul 10 23:22:41.678385 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:41.678284 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-lg7dx_2b6daf41-2288-4cbf-ac15-2e333b6870ce/kube-storage-version-migrator-operator/1.log" Jul 10 23:22:41.679038 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:41.679016 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-lg7dx_2b6daf41-2288-4cbf-ac15-2e333b6870ce/kube-storage-version-migrator-operator/0.log" Jul 10 23:22:42.900035 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:42.900000 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/kube-multus-additional-cni-plugins/0.log" Jul 10 23:22:42.929461 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:42.929423 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/egress-router-binary-copy/0.log" Jul 10 23:22:42.959458 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:42.959427 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/cni-plugins/0.log" Jul 10 23:22:42.986669 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:42.986637 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/bond-cni-plugin/0.log" Jul 10 23:22:43.024028 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:43.023990 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/routeoverride-cni/0.log" Jul 10 23:22:43.063855 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:43.063821 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/whereabouts-cni-bincopy/0.log" Jul 10 23:22:43.109513 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:43.109484 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p5d8f_2f6886d0-03e4-4074-9a3d-6105b77c91cc/whereabouts-cni/0.log" Jul 10 23:22:43.382150 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:43.382112 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fggm7_0d9861af-65ee-4e7b-94d6-1c44887a2d68/kube-multus/0.log" Jul 10 23:22:43.447327 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:43.447290 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-28nzx_931fa3c6-353f-43b5-9016-bfae335b8cc7/network-metrics-daemon/0.log" Jul 10 23:22:43.473038 ip-10-0-138-188 kubenswrapper[2551]: I0710 23:22:43.473010 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-28nzx_931fa3c6-353f-43b5-9016-bfae335b8cc7/kube-rbac-proxy/0.log"