Jul 10 20:37:44.547933 ip-10-0-136-104 systemd[1]: Starting Kubernetes Kubelet... Jul 10 20:37:44.895876 ip-10-0-136-104 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:37:44.895876 ip-10-0-136-104 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 20:37:44.895876 ip-10-0-136-104 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:37:44.895876 ip-10-0-136-104 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 20:37:44.895876 ip-10-0-136-104 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:37:44.897579 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.897498 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 20:37:44.901759 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901743 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:37:44.901759 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901758 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901762 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901765 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901774 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901777 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901780 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901783 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901786 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901789 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901792 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901796 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901800 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901803 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901806 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901810 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901813 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901816 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901818 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901821 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:37:44.901818 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901824 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901827 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901831 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901833 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901836 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901838 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901841 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901844 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901846 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901849 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901852 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901854 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901865 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901868 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901871 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901874 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901876 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901882 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901885 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901888 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:37:44.902264 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901891 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901893 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901896 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901898 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901901 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901903 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901906 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901908 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901911 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901913 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901916 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901918 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901920 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901924 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901927 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901930 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901935 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901939 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901942 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:37:44.902750 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901945 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901947 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901950 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901953 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901955 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901958 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901960 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901963 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901965 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901967 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901970 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901973 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901976 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901978 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901981 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901983 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901986 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901988 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901991 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901993 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:37:44.903206 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901996 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.901998 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902001 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902003 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902005 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902008 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902011 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902014 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902017 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902020 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902022 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902025 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902027 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902031 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902033 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902035 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902038 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902040 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902043 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902045 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:37:44.903965 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902761 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902767 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902771 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902774 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902777 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902780 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902782 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902785 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902787 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902790 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902793 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902796 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902799 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902802 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902807 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902810 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902813 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902815 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902818 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902821 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:37:44.904525 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902824 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902826 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902829 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902832 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902834 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902837 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902839 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902842 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902844 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902848 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902852 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902855 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902858 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902861 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902863 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902866 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902868 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902872 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902875 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:37:44.905028 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902877 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902879 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902882 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902884 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902887 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902890 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902893 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902895 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902898 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902901 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902903 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902906 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902908 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902913 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902916 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902919 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902922 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902925 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902927 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902930 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:37:44.905496 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902933 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902936 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902938 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902941 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902943 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902946 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902948 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902951 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902953 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902956 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902958 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902964 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902966 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902969 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902971 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902974 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902977 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902980 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902982 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902985 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:37:44.905984 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902987 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902990 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902992 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902995 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.902997 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903001 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903004 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903006 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903009 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903011 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903014 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903016 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903019 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903022 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903024 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903026 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903029 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903032 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903034 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:37:44.906457 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.903036 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906126 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906137 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906141 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906145 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906149 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906153 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906160 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906163 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906168 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906171 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906174 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906177 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906179 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906182 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906186 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906189 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906192 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906195 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:37:44.906947 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906200 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906203 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906206 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906209 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906212 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906215 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906218 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906220 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906223 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906226 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906228 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906231 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906236 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906238 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906241 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906244 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906246 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906249 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906252 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:37:44.907411 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906256 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906258 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906261 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906263 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906266 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906271 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906274 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906276 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906279 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906282 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906285 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906288 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906291 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906293 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906296 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906298 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906301 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906304 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906309 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906312 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:37:44.907885 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906316 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906319 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906321 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906324 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906326 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906329 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906332 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906334 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906337 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906340 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906344 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906347 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906351 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906354 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906357 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906359 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906362 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906364 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906367 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906370 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:37:44.908355 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906372 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906378 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906384 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906386 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906389 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906392 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906394 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906397 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906399 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906402 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906405 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906408 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906411 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906413 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906416 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906421 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906423 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906426 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906429 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906431 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:37:44.908871 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906434 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.906437 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906632 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906792 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906802 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906807 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906813 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906817 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906830 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906834 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906838 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906841 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906844 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906847 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906851 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906854 2577 flags.go:64] FLAG: --cgroup-root="" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906856 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906859 2577 flags.go:64] FLAG: --client-ca-file="" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906862 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906865 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906869 2577 flags.go:64] FLAG: --cluster-domain="" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906872 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906875 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906878 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 20:37:44.909339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906882 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906886 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906889 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906891 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906895 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906898 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906901 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906904 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906907 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906910 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906914 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906918 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906921 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906924 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906928 2577 flags.go:64] FLAG: --enable-server="true" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906931 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906936 2577 flags.go:64] FLAG: --event-burst="100" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906939 2577 flags.go:64] FLAG: --event-qps="50" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906942 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906945 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906948 2577 flags.go:64] FLAG: --eviction-hard="" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906952 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906955 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906958 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906962 2577 flags.go:64] FLAG: --eviction-soft="" Jul 10 20:37:44.909949 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906965 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906968 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906970 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906973 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906976 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906979 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906982 2577 flags.go:64] FLAG: --feature-gates="" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906985 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906988 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906991 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906995 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.906997 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907000 2577 flags.go:64] FLAG: --help="false" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907004 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-136-104.ec2.internal" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907007 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907010 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907013 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907017 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907021 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907024 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907027 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907030 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907034 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 20:37:44.910522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907036 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907040 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907043 2577 flags.go:64] FLAG: --kube-reserved="" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907046 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907048 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907051 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907054 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907057 2577 flags.go:64] FLAG: --lock-file="" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907060 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907063 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907065 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907071 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907074 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907076 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907079 2577 flags.go:64] FLAG: --logging-format="text" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907082 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907086 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907088 2577 flags.go:64] FLAG: --manifest-url="" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907091 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907096 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907099 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907103 2577 flags.go:64] FLAG: --max-pods="110" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907106 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907109 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907112 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 20:37:44.911119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907115 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907118 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907121 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907124 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907133 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907136 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907139 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907142 2577 flags.go:64] FLAG: --pod-cidr="" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907145 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907150 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907152 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907155 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907158 2577 flags.go:64] FLAG: --port="10250" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907161 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907164 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fb196aa44da3e219" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907167 2577 flags.go:64] FLAG: --qos-reserved="" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907169 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907172 2577 flags.go:64] FLAG: --register-node="true" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907175 2577 flags.go:64] FLAG: --register-with-taints="" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907178 2577 flags.go:64] FLAG: --registry-burst="10" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907181 2577 flags.go:64] FLAG: --registry-qps="5" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907185 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907187 2577 flags.go:64] FLAG: --reserved-memory="" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907191 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907194 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 20:37:44.911718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907197 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907199 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907202 2577 flags.go:64] FLAG: --runonce="false" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907204 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907207 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907210 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907213 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907216 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907219 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907222 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907225 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907228 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907231 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907234 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907236 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907239 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907242 2577 flags.go:64] FLAG: --system-cgroups="" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907245 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907250 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907253 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907256 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907260 2577 flags.go:64] FLAG: --tls-min-version="" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907263 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907265 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907268 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 20:37:44.912330 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907271 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907274 2577 flags.go:64] FLAG: --v="2" Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907277 2577 flags.go:64] FLAG: --version="false" Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907281 2577 flags.go:64] FLAG: --vmodule="" Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907286 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.907289 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907423 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907427 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907430 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907433 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907436 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907439 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907442 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907445 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907448 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907450 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907453 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907456 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907459 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907461 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907465 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:37:44.912943 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907468 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907471 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907474 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907476 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907479 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907481 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907484 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907486 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907489 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907493 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907497 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907499 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907502 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907505 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907508 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907515 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907518 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907521 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907523 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907526 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:37:44.913562 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907528 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907531 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907534 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907537 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907539 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907542 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907544 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907547 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907550 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907552 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907555 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907557 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907560 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907562 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907565 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907568 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907570 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907573 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907575 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907577 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:37:44.914410 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907580 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907582 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907585 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907587 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907590 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907592 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907594 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907598 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907601 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907604 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907607 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907609 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907612 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907614 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907617 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907619 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907622 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907624 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907627 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:37:44.915276 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907631 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907634 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907638 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907641 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907644 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907663 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907667 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907670 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907672 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907675 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907678 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907681 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907683 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907686 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907688 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907691 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907693 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907696 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907699 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907701 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:37:44.916081 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907706 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907709 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907712 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907715 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.907717 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.908259 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.915714 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.915738 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.915765 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.915774 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915863 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915871 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915876 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915881 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915887 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:37:44.916975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915891 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915896 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915900 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915905 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915909 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915913 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915917 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915922 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915926 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915930 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915934 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915938 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915943 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915947 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915952 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915956 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915960 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915965 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915969 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:37:44.917487 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915975 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915981 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915986 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915991 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.915995 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916003 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916007 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916012 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916016 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916021 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916025 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916029 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916033 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916037 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916042 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916046 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916050 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916055 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916059 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916063 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:37:44.918032 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916068 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916072 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916076 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916080 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916083 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916088 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916092 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916096 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916100 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916104 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916109 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916113 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916117 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916122 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916126 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916130 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916135 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916139 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916145 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916149 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:37:44.918668 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916153 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916157 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916161 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916165 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916170 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916174 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916179 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916183 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916187 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916191 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916195 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916199 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916204 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916208 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916212 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916219 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916225 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916230 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916234 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:37:44.919621 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916239 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916243 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916247 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916252 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916256 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916260 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916264 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916269 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916274 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916278 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916282 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916287 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916292 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916296 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916301 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:37:44.920141 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916307 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.916315 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916493 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916502 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916508 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916512 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916517 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916522 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916527 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916531 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916535 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916540 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916544 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916549 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:37:44.920631 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916553 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916557 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916561 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916565 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916570 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916574 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916578 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916582 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916586 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916591 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916595 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916601 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916608 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916613 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916617 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916622 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916632 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916637 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916641 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:37:44.921176 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916645 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916649 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916672 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916676 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916679 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916683 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916686 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916690 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916694 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916698 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916702 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916707 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916711 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916715 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916719 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916723 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916727 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916732 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916736 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916740 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:37:44.921720 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916744 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916748 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916752 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916757 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916761 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916765 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916769 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916773 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916777 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916781 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916794 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916799 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916803 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916807 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916811 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916815 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916819 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916823 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916827 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916832 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:37:44.922246 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916836 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916840 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916845 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916851 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916857 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916861 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916866 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916870 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916875 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916879 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916883 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916887 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916891 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916896 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916900 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916905 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916909 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916913 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916917 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:37:44.922749 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916921 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916926 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916930 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916933 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916939 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916944 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916948 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916952 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:44.916956 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.916964 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.917124 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.920242 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.921409 2577 server.go:1019] "Starting client certificate rotation" Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.921517 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 20:37:44.923241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.922441 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 20:37:44.923934 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.923919 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 20:37:44.925097 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.925077 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 20:37:44.940009 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.939988 2577 log.go:25] "Validated CRI v1 runtime API" Jul 10 20:37:44.942011 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.941997 2577 log.go:25] "Validated CRI v1 image API" Jul 10 20:37:44.942834 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.942811 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 20:37:44.945218 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.945186 2577 fs.go:135] Filesystem UUIDs: map[4cb6bcda-99bd-4422-b060-9637ce7fb9c0:/dev/nvme0n1p3 752ba0d4-e4c4-44ae-9243-9c018c4eaf73:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 10 20:37:44.945295 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.945216 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 20:37:44.952303 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.952198 2577 manager.go:216] Machine: {Timestamp:2026-07-10 20:37:44.951095339 +0000 UTC m=+0.299404661 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100090 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2511a18fcd17aca6fbac31192abcf9 SystemUUID:ec2511a1-8fcd-17ac-a6fb-ac31192abcf9 BootID:5ae45592-a717-41b1-b9ea-02afec646085 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:88:a3:8a:6d:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:88:a3:8a:6d:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:63:90:36:0e:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 20:37:44.952303 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.952295 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 20:37:44.952434 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.952362 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 20:37:44.953790 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.953775 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 20:37:44.954927 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.954893 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 20:37:44.955058 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.954931 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 20:37:44.955511 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.955502 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 20:37:44.955547 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.955513 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 20:37:44.955547 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.955520 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 20:37:44.956181 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.956170 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 20:37:44.957289 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.957279 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 20:37:44.960676 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960649 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 10 20:37:44.960831 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960822 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 20:37:44.960869 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960853 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 10 20:37:44.960869 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960863 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 20:37:44.960920 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960881 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 20:37:44.960920 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960889 2577 kubelet.go:398] "Adding apiserver pod source" Jul 10 20:37:44.960920 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.960898 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 20:37:44.961199 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.961187 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 20:37:44.961251 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.961205 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 20:37:44.963248 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.963234 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 20:37:44.964866 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.964853 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 20:37:44.964914 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.964884 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 20:37:44.965036 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965026 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 20:37:44.965067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965041 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 20:37:44.965067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965047 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 20:37:44.965067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965052 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 20:37:44.965067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965060 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 20:37:44.965067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965065 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 20:37:44.965649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965640 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 20:37:44.965649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965648 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 20:37:44.965732 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965670 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 20:37:44.965732 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965676 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 20:37:44.965732 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965685 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 20:37:44.965732 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.965694 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 20:37:44.966585 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.966576 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 20:37:44.966585 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.966585 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 20:37:44.969530 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.969516 2577 server.go:1268] "Started kubelet" Jul 10 20:37:44.970210 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.970192 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 20:37:44.970425 ip-10-0-136-104 systemd[1]: Started Kubernetes Kubelet. Jul 10 20:37:44.971806 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.971792 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 10 20:37:44.972522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.972498 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 20:37:44.972595 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.972535 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 20:37:44.973875 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.973863 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 20:37:44.974702 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.974674 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-klzp6" Jul 10 20:37:44.976311 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.976294 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 20:37:44.976426 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.976411 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 20:37:44.976613 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.976593 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 20:37:44.976814 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.976798 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 20:37:44.976894 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.976825 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 20:37:44.976949 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.976931 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:44.977904 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.977885 2577 factory.go:55] Registering systemd factory Jul 10 20:37:44.977904 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.977909 2577 factory.go:223] Registration of the systemd container factory successfully Jul 10 20:37:44.978089 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978074 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 20:37:44.978155 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978092 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 20:37:44.978155 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978102 2577 factory.go:153] Registering CRI-O factory Jul 10 20:37:44.978155 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978117 2577 factory.go:223] Registration of the crio container factory successfully Jul 10 20:37:44.978278 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978164 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 20:37:44.978278 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978186 2577 factory.go:103] Registering Raw factory Jul 10 20:37:44.978278 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978199 2577 manager.go:1196] Started watching for new ooms in manager Jul 10 20:37:44.978278 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978205 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 10 20:37:44.978278 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978212 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 10 20:37:44.978631 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978614 2577 manager.go:349] Starting recovery of all containers Jul 10 20:37:44.978848 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.978825 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 20:37:44.980533 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.980501 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 20:37:44.980690 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.980650 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 20:37:44.981798 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.980589 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-104.ec2.internal.18c10820efbc4de0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-104.ec2.internal,UID:ip-10-0-136-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-104.ec2.internal,},FirstTimestamp:2026-07-10 20:37:44.96949808 +0000 UTC m=+0.317807401,LastTimestamp:2026-07-10 20:37:44.96949808 +0000 UTC m=+0.317807401,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-104.ec2.internal,}" Jul 10 20:37:44.982765 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.982742 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-klzp6" Jul 10 20:37:44.985736 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:44.985695 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 20:37:44.991529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.991512 2577 manager.go:354] Recovery completed Jul 10 20:37:44.996046 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.996034 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:44.997821 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.997805 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:44.997894 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.997839 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:44.997894 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.997854 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:44.998484 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.998458 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 20:37:44.998484 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.998481 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 20:37:44.998629 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.998494 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 10 20:37:44.998629 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.998470 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:44.999301 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.999282 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:44.999383 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.999317 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:44.999383 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:44.999332 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:45.000086 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.000007 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-104.ec2.internal.18c10820f16c7e48 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-104.ec2.internal,UID:ip-10-0-136-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-104.ec2.internal,},FirstTimestamp:2026-07-10 20:37:44.997822024 +0000 UTC m=+0.346131352,LastTimestamp:2026-07-10 20:37:44.997822024 +0000 UTC m=+0.346131352,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-104.ec2.internal,}" Jul 10 20:37:45.000460 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.000448 2577 policy_none.go:49] "None policy: Start" Jul 10 20:37:45.000507 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.000464 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 20:37:45.000507 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.000474 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 20:37:45.002366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.002353 2577 policy_none.go:47] "Start" Jul 10 20:37:45.039313 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039185 2577 manager.go:337] "Starting Device Plugin manager" Jul 10 20:37:45.039418 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.039343 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 20:37:45.039418 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039355 2577 server.go:87] "Starting device plugin registration server" Jul 10 20:37:45.039597 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039585 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 20:37:45.039649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039597 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 20:37:45.039710 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039700 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 20:37:45.039817 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039804 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 20:37:45.039817 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.039817 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 20:37:45.040330 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.040307 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 20:37:45.040413 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.040363 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.140162 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.140129 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:45.142244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.141142 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:45.142244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.141173 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:45.142244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.141211 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:45.142244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.141272 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.151174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.151132 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.151174 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.151155 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-104.ec2.internal\": node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.156689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.156643 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 20:37:45.157940 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.157920 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 20:37:45.158052 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.157962 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 20:37:45.158052 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.158028 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 20:37:45.158143 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.158067 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 20:37:45.159626 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.159606 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:37:45.162520 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.162504 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.258594 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.258560 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal"] Jul 10 20:37:45.258675 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.258637 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:45.260886 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.260870 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:45.260974 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.260900 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:45.260974 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.260910 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:45.263041 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.263024 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.263360 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.263346 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:45.263486 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.263472 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.263543 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.263512 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:45.264226 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.264203 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:45.264226 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.264222 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:45.264368 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.264232 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:45.264368 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.264245 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:45.264368 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.264258 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:45.264368 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.264246 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:45.266968 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.266951 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.267071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.266976 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:37:45.267822 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.267801 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:37:45.267910 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.267832 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:37:45.267910 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.267865 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:37:45.291413 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.291390 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-104.ec2.internal\" not found" node="ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.295930 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.295914 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-104.ec2.internal\" not found" node="ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.364054 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.364030 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.380611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.380585 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d8de27905a457ddc122bd67be75fc31e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-104.ec2.internal\" (UID: \"d8de27905a457ddc122bd67be75fc31e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.380727 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.380612 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/13ed74dbe625057f78bd09ff505253e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"13ed74dbe625057f78bd09ff505253e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.380727 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.380630 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13ed74dbe625057f78bd09ff505253e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"13ed74dbe625057f78bd09ff505253e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.464263 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.464165 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.480847 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.480799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d8de27905a457ddc122bd67be75fc31e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-104.ec2.internal\" (UID: \"d8de27905a457ddc122bd67be75fc31e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.480847 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.480853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/13ed74dbe625057f78bd09ff505253e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"13ed74dbe625057f78bd09ff505253e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.481014 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.480872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13ed74dbe625057f78bd09ff505253e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"13ed74dbe625057f78bd09ff505253e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.481014 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.480924 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d8de27905a457ddc122bd67be75fc31e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-104.ec2.internal\" (UID: \"d8de27905a457ddc122bd67be75fc31e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.481014 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.480949 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13ed74dbe625057f78bd09ff505253e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"13ed74dbe625057f78bd09ff505253e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.481014 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.480924 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/13ed74dbe625057f78bd09ff505253e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal\" (UID: \"13ed74dbe625057f78bd09ff505253e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.570853 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.570818 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.592979 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.592949 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.598341 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.598322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" Jul 10 20:37:45.671802 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.671757 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.772862 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.772829 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.875203 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.875154 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.924942 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.924912 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 20:37:45.925487 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.925066 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 20:37:45.976131 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:45.976094 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:45.987057 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.987018 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 20:32:45 +0000 UTC" deadline="2028-03-12 18:11:24.44780998 +0000 UTC" Jul 10 20:37:45.987057 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:45.987053 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14661h33m38.46076074s" Jul 10 20:37:46.020487 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.020458 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:37:46.076329 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:46.076259 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:46.126880 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.126851 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:37:46.176636 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:46.176607 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:46.239392 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:46.239356 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8de27905a457ddc122bd67be75fc31e.slice/crio-2ba13cb4a88251e4f54a98b285a25c9ee72d83471621039ed5f4bf668deb95b7 WatchSource:0}: Error finding container 2ba13cb4a88251e4f54a98b285a25c9ee72d83471621039ed5f4bf668deb95b7: Status 404 returned error can't find the container with id 2ba13cb4a88251e4f54a98b285a25c9ee72d83471621039ed5f4bf668deb95b7 Jul 10 20:37:46.249269 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.249249 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:37:46.254975 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:46.254950 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ed74dbe625057f78bd09ff505253e2.slice/crio-a83de71c5c68d51929a8ac042120c34b16b2e2a0ae9f2b0465a1a0c95e0ebb37 WatchSource:0}: Error finding container a83de71c5c68d51929a8ac042120c34b16b2e2a0ae9f2b0465a1a0c95e0ebb37: Status 404 returned error can't find the container with id a83de71c5c68d51929a8ac042120c34b16b2e2a0ae9f2b0465a1a0c95e0ebb37 Jul 10 20:37:46.277688 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:46.277643 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-104.ec2.internal\" not found" Jul 10 20:37:46.299898 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.299872 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:37:46.380440 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.380336 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" Jul 10 20:37:46.392373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.392349 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 20:37:46.393262 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.393248 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" Jul 10 20:37:46.402177 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.402161 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 20:37:46.773110 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.773073 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:37:46.964847 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.964810 2577 apiserver.go:52] "Watching apiserver" Jul 10 20:37:46.977111 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.977084 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 20:37:46.977320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.977289 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 20:37:46.979288 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.979263 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z","openshift-dns/node-resolver-mn55h","openshift-image-registry/node-ca-p9p6r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal","openshift-multus/multus-additional-cni-plugins-zhchj","openshift-multus/multus-c5g85","kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal","openshift-cluster-node-tuning-operator/tuned-c4fmw","openshift-multus/network-metrics-daemon-fbz74","openshift-network-diagnostics/network-check-target-tqvmr","openshift-network-operator/iptables-alerter-fhvv8","openshift-ovn-kubernetes/ovnkube-node-zh7ps","kube-system/global-pull-secret-syncer-nbn2w","kube-system/konnectivity-agent-jvmm8"] Jul 10 20:37:46.982164 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.982142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:46.982282 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:46.982220 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:37:46.984381 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.984358 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:46.986935 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.986908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:46.987213 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.987191 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4zph8\"" Jul 10 20:37:46.987578 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.987558 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 20:37:46.987666 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.987601 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 20:37:46.987993 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.987966 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 20:37:46.989091 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.989070 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:46.989256 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.989234 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 20:37:46.990174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.990105 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tbns7\"" Jul 10 20:37:46.990817 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.990390 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 20:37:46.990817 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.990649 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 20:37:46.991855 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.991835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:46.992764 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.992726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 20:37:46.993440 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.993062 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 20:37:46.993440 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.993177 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 20:37:46.993608 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.993443 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87bjh\"" Jul 10 20:37:46.996058 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.996038 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c5g85" Jul 10 20:37:46.996390 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.996372 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 20:37:46.997244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.997220 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 20:37:46.997353 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.997331 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 20:37:46.997437 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.997420 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4f75q\"" Jul 10 20:37:46.997558 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.997537 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 20:37:46.997624 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.997346 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 20:37:46.999452 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.999430 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 20:37:46.999671 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.999654 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:46.999754 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:46.999675 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d4hb4\"" Jul 10 20:37:47.001702 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.001683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.002444 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.002423 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 20:37:47.002684 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.002667 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 20:37:47.003004 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.002980 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xbdqz\"" Jul 10 20:37:47.004292 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.004273 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 20:37:47.004499 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.004478 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:37:47.004696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.004682 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-975lm\"" Jul 10 20:37:47.006227 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.006206 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.006350 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.006302 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:47.008774 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.008754 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:47.008871 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.008822 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:47.009671 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.009653 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7q2s" Jul 10 20:37:47.011414 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.011393 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.014046 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.014025 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 20:37:47.014160 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.014144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.014722 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.014701 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 20:37:47.014850 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.014752 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:37:47.014850 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.014777 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sgwbj\"" Jul 10 20:37:47.016540 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.016511 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f2hxl\"" Jul 10 20:37:47.016827 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.016806 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 20:37:47.017548 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.017527 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 20:37:47.017920 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.017900 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 20:37:47.018443 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.018425 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 20:37:47.018535 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.018453 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 20:37:47.018691 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.018670 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 20:37:47.022313 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.022290 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7q2s" Jul 10 20:37:47.080316 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.080294 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 20:37:47.089507 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.089628 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089520 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xklf\" (UniqueName: \"kubernetes.io/projected/92929035-69df-4c3f-8e04-9d272ae67909-kube-api-access-4xklf\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.089628 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.089772 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089648 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-cni-multus\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.089772 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-tuned\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.089772 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-node-log\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.089772 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-log-socket\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.089975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-env-overrides\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.089975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089840 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovn-node-metrics-cert\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.089975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92929035-69df-4c3f-8e04-9d272ae67909-hosts-file\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.089975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.089975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089930 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:47.089975 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089954 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.089978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-netns\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-kubelet\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-systemd-units\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-systemd\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-etc-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-cnibin\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090226 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd11744a-1747-4d79-861b-4197fa608a0b-cni-binary-copy\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090251 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-multus-certs\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-iptables-alerter-script\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090311 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-kubernetes\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77pqm\" (UniqueName: \"kubernetes.io/projected/92aabedc-2b21-4989-8acb-5179e6902cb8-kube-api-access-77pqm\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-kubelet\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-os-release\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-host-slash\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8xmv\" (UniqueName: \"kubernetes.io/projected/71a681fe-80db-4415-acc7-c3c0417e4157-kube-api-access-j8xmv\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090495 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-cni-netd\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwq6c\" (UniqueName: \"kubernetes.io/projected/53756faf-a958-4c95-a856-ad2593be1e1f-kube-api-access-qwq6c\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-k8s-cni-cncf-io\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090596 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-cni-bin\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090622 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-socket-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.090674 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090652 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysconfig\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090680 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysctl-d\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090721 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-host\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-slash\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-run-ovn-kubernetes\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/decac6e6-5885-4081-8f10-041e44c6a00e-kubelet-config\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-cnibin\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysctl-conf\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/decac6e6-5885-4081-8f10-041e44c6a00e-dbus\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-system-cni-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090971 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-cni-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.090995 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-hostroot\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-etc-kubernetes\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091045 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcrzr\" (UniqueName: \"kubernetes.io/projected/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-kube-api-access-xcrzr\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-device-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091117 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfzss\" (UniqueName: \"kubernetes.io/projected/a8f8dde8-d179-4c65-b586-58d76062a729-kube-api-access-mfzss\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.091320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-var-lib-kubelet\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-var-lib-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovnkube-script-lib\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92929035-69df-4c3f-8e04-9d272ae67909-tmp-dir\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-os-release\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091390 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/71a681fe-80db-4415-acc7-c3c0417e4157-serviceca\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9f059320-52a4-4f1a-b987-0a60ed0159d7-agent-certs\") pod \"konnectivity-agent-jvmm8\" (UID: \"9f059320-52a4-4f1a-b987-0a60ed0159d7\") " pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091460 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-sys\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65z5w\" (UniqueName: \"kubernetes.io/projected/4c9d3179-349e-4c59-a906-28e4fbccffb8-kube-api-access-65z5w\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091513 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-system-cni-dir\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091543 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-conf-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091591 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2spgj\" (UniqueName: \"kubernetes.io/projected/dd11744a-1747-4d79-861b-4197fa608a0b-kube-api-access-2spgj\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-sys-fs\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091638 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-modprobe-d\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.092174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091670 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-lib-modules\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091710 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-registration-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091759 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-run\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92aabedc-2b21-4989-8acb-5179e6902cb8-tmp\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091813 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovnkube-config\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091924 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-run-netns\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091949 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-ovn\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd11744a-1747-4d79-861b-4197fa608a0b-multus-daemon-config\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.091999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lft5t\" (UniqueName: \"kubernetes.io/projected/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-kube-api-access-lft5t\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.092023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9f059320-52a4-4f1a-b987-0a60ed0159d7-konnectivity-ca\") pod \"konnectivity-agent-jvmm8\" (UID: \"9f059320-52a4-4f1a-b987-0a60ed0159d7\") " pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.092048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-systemd\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.092077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-cni-bin\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.092101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-socket-dir-parent\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.093000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.092124 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71a681fe-80db-4415-acc7-c3c0417e4157-host\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.163386 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.163304 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" event={"ID":"13ed74dbe625057f78bd09ff505253e2","Type":"ContainerStarted","Data":"a83de71c5c68d51929a8ac042120c34b16b2e2a0ae9f2b0465a1a0c95e0ebb37"} Jul 10 20:37:47.165241 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.165206 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" event={"ID":"d8de27905a457ddc122bd67be75fc31e","Type":"ContainerStarted","Data":"2ba13cb4a88251e4f54a98b285a25c9ee72d83471621039ed5f4bf668deb95b7"} Jul 10 20:37:47.192827 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.192827 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xklf\" (UniqueName: \"kubernetes.io/projected/92929035-69df-4c3f-8e04-9d272ae67909-kube-api-access-4xklf\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-cni-multus\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-tuned\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-node-log\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192928 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-log-socket\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-env-overrides\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-cni-multus\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.192998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovn-node-metrics-cert\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92929035-69df-4c3f-8e04-9d272ae67909-hosts-file\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.193082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193134 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-netns\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-kubelet\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-systemd-units\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193210 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-systemd\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-etc-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193289 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-cnibin\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd11744a-1747-4d79-861b-4197fa608a0b-cni-binary-copy\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-multus-certs\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-iptables-alerter-script\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193401 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-kubernetes\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77pqm\" (UniqueName: \"kubernetes.io/projected/92aabedc-2b21-4989-8acb-5179e6902cb8-kube-api-access-77pqm\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-kubelet\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-os-release\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-host-slash\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.193768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8xmv\" (UniqueName: \"kubernetes.io/projected/71a681fe-80db-4415-acc7-c3c0417e4157-kube-api-access-j8xmv\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-cni-netd\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193619 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193624 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwq6c\" (UniqueName: \"kubernetes.io/projected/53756faf-a958-4c95-a856-ad2593be1e1f-kube-api-access-qwq6c\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193650 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-k8s-cni-cncf-io\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193708 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-systemd\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193712 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-k8s-cni-cncf-io\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-cni-bin\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193776 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-etc-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-socket-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysconfig\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysctl-d\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-host\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-slash\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193922 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-run-ovn-kubernetes\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-run-ovn-kubernetes\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.194689 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/decac6e6-5885-4081-8f10-041e44c6a00e-kubelet-config\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.193997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-cnibin\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194004 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/decac6e6-5885-4081-8f10-041e44c6a00e-kubelet-config\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysctl-conf\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194060 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/decac6e6-5885-4081-8f10-041e44c6a00e-dbus\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-system-cni-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-cni-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194165 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-hostroot\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-etc-kubernetes\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcrzr\" (UniqueName: \"kubernetes.io/projected/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-kube-api-access-xcrzr\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194247 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-device-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194275 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfzss\" (UniqueName: \"kubernetes.io/projected/a8f8dde8-d179-4c65-b586-58d76062a729-kube-api-access-mfzss\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-var-lib-kubelet\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194304 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194330 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-var-lib-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-cnibin\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194356 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.195611 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovnkube-script-lib\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92929035-69df-4c3f-8e04-9d272ae67909-tmp-dir\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194464 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-os-release\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194477 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-node-log\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/71a681fe-80db-4415-acc7-c3c0417e4157-serviceca\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194523 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9f059320-52a4-4f1a-b987-0a60ed0159d7-agent-certs\") pod \"konnectivity-agent-jvmm8\" (UID: \"9f059320-52a4-4f1a-b987-0a60ed0159d7\") " pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-sys\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65z5w\" (UniqueName: \"kubernetes.io/projected/4c9d3179-349e-4c59-a906-28e4fbccffb8-kube-api-access-65z5w\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-system-cni-dir\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-conf-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2spgj\" (UniqueName: \"kubernetes.io/projected/dd11744a-1747-4d79-861b-4197fa608a0b-kube-api-access-2spgj\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-sys-fs\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-modprobe-d\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194713 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-cni-bin\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-lib-modules\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.196468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-registration-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194811 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194835 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-lib-modules\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-run\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194882 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysconfig\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194897 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-log-socket\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/71a681fe-80db-4415-acc7-c3c0417e4157-serviceca\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194954 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovnkube-script-lib\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.194971 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-socket-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195011 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-host\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195047 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-slash\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195083 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysctl-conf\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195125 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-multus-certs\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/decac6e6-5885-4081-8f10-041e44c6a00e-dbus\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195263 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-system-cni-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195286 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-env-overrides\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-cni-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195398 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-hostroot\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.197399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195446 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-etc-kubernetes\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195619 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd11744a-1747-4d79-861b-4197fa608a0b-cni-binary-copy\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195709 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-run-netns\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195748 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-sys\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195814 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195824 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-kubelet\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-host-slash\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-device-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195965 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-system-cni-dir\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-kubernetes\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92929035-69df-4c3f-8e04-9d272ae67909-hosts-file\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196147 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-cni-netd\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196152 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-cnibin\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196207 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-var-lib-kubelet\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-var-lib-openvswitch\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-modprobe-d\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196542 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-conf-dir\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.198373 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.195082 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-sysctl-d\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196691 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53756faf-a958-4c95-a856-ad2593be1e1f-os-release\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196777 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-os-release\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196850 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-sys-fs\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92929035-69df-4c3f-8e04-9d272ae67909-tmp-dir\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196961 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-registration-dir\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.196978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-host-var-lib-kubelet\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197017 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-systemd-units\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-run\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197115 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8f8dde8-d179-4c65-b586-58d76062a729-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92aabedc-2b21-4989-8acb-5179e6902cb8-tmp\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-iptables-alerter-script\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovnkube-config\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197207 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53756faf-a958-4c95-a856-ad2593be1e1f-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-run-netns\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-ovn\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.199366 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.197359 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd11744a-1747-4d79-861b-4197fa608a0b-multus-daemon-config\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lft5t\" (UniqueName: \"kubernetes.io/projected/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-kube-api-access-lft5t\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197413 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-run-netns\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.197426 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:37:47.697404873 +0000 UTC m=+3.045714201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197515 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9f059320-52a4-4f1a-b987-0a60ed0159d7-konnectivity-ca\") pod \"konnectivity-agent-jvmm8\" (UID: \"9f059320-52a4-4f1a-b987-0a60ed0159d7\") " pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197572 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-systemd\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197607 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-run-ovn\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-cni-bin\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-socket-dir-parent\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71a681fe-80db-4415-acc7-c3c0417e4157-host\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-systemd\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197723 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c9d3179-349e-4c59-a906-28e4fbccffb8-host-cni-bin\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.197704 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197782 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd11744a-1747-4d79-861b-4197fa608a0b-multus-socket-dir-parent\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.197789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71a681fe-80db-4415-acc7-c3c0417e4157-host\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.197810 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:47.697791786 +0000 UTC m=+3.046101099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:47.200302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.198107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd11744a-1747-4d79-861b-4197fa608a0b-multus-daemon-config\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.201242 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.198312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9f059320-52a4-4f1a-b987-0a60ed0159d7-konnectivity-ca\") pod \"konnectivity-agent-jvmm8\" (UID: \"9f059320-52a4-4f1a-b987-0a60ed0159d7\") " pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.201242 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.198323 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovnkube-config\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.201242 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.198812 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9f059320-52a4-4f1a-b987-0a60ed0159d7-agent-certs\") pod \"konnectivity-agent-jvmm8\" (UID: \"9f059320-52a4-4f1a-b987-0a60ed0159d7\") " pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.201242 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.200795 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92aabedc-2b21-4989-8acb-5179e6902cb8-tmp\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.201242 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.200829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92aabedc-2b21-4989-8acb-5179e6902cb8-etc-tuned\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.201242 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.201082 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c9d3179-349e-4c59-a906-28e4fbccffb8-ovn-node-metrics-cert\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.205516 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.205041 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:37:47.205516 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.205079 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:37:47.205516 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.205107 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:47.205516 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.205176 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:47.705159872 +0000 UTC m=+3.053469202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:47.205940 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.205561 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcrzr\" (UniqueName: \"kubernetes.io/projected/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-kube-api-access-xcrzr\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.206913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.206888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xklf\" (UniqueName: \"kubernetes.io/projected/92929035-69df-4c3f-8e04-9d272ae67909-kube-api-access-4xklf\") pod \"node-resolver-mn55h\" (UID: \"92929035-69df-4c3f-8e04-9d272ae67909\") " pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.209224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.209153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfzss\" (UniqueName: \"kubernetes.io/projected/a8f8dde8-d179-4c65-b586-58d76062a729-kube-api-access-mfzss\") pod \"aws-ebs-csi-driver-node-2xv8z\" (UID: \"a8f8dde8-d179-4c65-b586-58d76062a729\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.209224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.209174 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwq6c\" (UniqueName: \"kubernetes.io/projected/53756faf-a958-4c95-a856-ad2593be1e1f-kube-api-access-qwq6c\") pod \"multus-additional-cni-plugins-zhchj\" (UID: \"53756faf-a958-4c95-a856-ad2593be1e1f\") " pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.210160 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.210134 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77pqm\" (UniqueName: \"kubernetes.io/projected/92aabedc-2b21-4989-8acb-5179e6902cb8-kube-api-access-77pqm\") pod \"tuned-c4fmw\" (UID: \"92aabedc-2b21-4989-8acb-5179e6902cb8\") " pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.210578 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.210557 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8xmv\" (UniqueName: \"kubernetes.io/projected/71a681fe-80db-4415-acc7-c3c0417e4157-kube-api-access-j8xmv\") pod \"node-ca-p9p6r\" (UID: \"71a681fe-80db-4415-acc7-c3c0417e4157\") " pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.211160 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.211118 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lft5t\" (UniqueName: \"kubernetes.io/projected/c1370c75-ebdf-40e1-9e7f-4ca40c55c869-kube-api-access-lft5t\") pod \"iptables-alerter-fhvv8\" (UID: \"c1370c75-ebdf-40e1-9e7f-4ca40c55c869\") " pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.211609 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.211583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65z5w\" (UniqueName: \"kubernetes.io/projected/4c9d3179-349e-4c59-a906-28e4fbccffb8-kube-api-access-65z5w\") pod \"ovnkube-node-zh7ps\" (UID: \"4c9d3179-349e-4c59-a906-28e4fbccffb8\") " pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.213301 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.213276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2spgj\" (UniqueName: \"kubernetes.io/projected/dd11744a-1747-4d79-861b-4197fa608a0b-kube-api-access-2spgj\") pod \"multus-c5g85\" (UID: \"dd11744a-1747-4d79-861b-4197fa608a0b\") " pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.300388 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.300316 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mn55h" Jul 10 20:37:47.310280 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.310255 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p9p6r" Jul 10 20:37:47.320337 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.320300 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" Jul 10 20:37:47.326298 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.326278 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zhchj" Jul 10 20:37:47.334232 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.334214 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c5g85" Jul 10 20:37:47.342077 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.342058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:37:47.349922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.349902 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" Jul 10 20:37:47.358811 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.358788 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fhvv8" Jul 10 20:37:47.365750 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.365721 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:37:47.703131 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.703030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:47.703131 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.703079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:47.703348 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.703217 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:47.703348 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.703268 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:47.703348 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.703297 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:48.703281916 +0000 UTC m=+4.051591249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:47.703348 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.703326 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:37:48.703309498 +0000 UTC m=+4.051618835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:47.804288 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:47.804241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:47.804491 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.804408 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:37:47.804491 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.804429 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:37:47.804491 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.804443 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:47.804662 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:47.804531 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:48.804510006 +0000 UTC m=+4.152819340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:48.025893 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.025831 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:32:47 +0000 UTC" deadline="2028-03-13 21:41:18.68141034 +0000 UTC" Jul 10 20:37:48.025893 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.025884 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14689h3m30.65552844s" Jul 10 20:37:48.159205 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.159168 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:48.159430 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.159301 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:48.159430 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.159364 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:48.159560 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.159509 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:48.345167 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.345133 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1370c75_ebdf_40e1_9e7f_4ca40c55c869.slice/crio-45f6359ad335784b91f82d14501dc1a602ecce2ec0348bc3b9943554e33bd1f7 WatchSource:0}: Error finding container 45f6359ad335784b91f82d14501dc1a602ecce2ec0348bc3b9943554e33bd1f7: Status 404 returned error can't find the container with id 45f6359ad335784b91f82d14501dc1a602ecce2ec0348bc3b9943554e33bd1f7 Jul 10 20:37:48.346949 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.346793 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a681fe_80db_4415_acc7_c3c0417e4157.slice/crio-185a42a4857cd83a3cd392822b90e4741c3337a575a894e41c3f596b1c44076c WatchSource:0}: Error finding container 185a42a4857cd83a3cd392822b90e4741c3337a575a894e41c3f596b1c44076c: Status 404 returned error can't find the container with id 185a42a4857cd83a3cd392822b90e4741c3337a575a894e41c3f596b1c44076c Jul 10 20:37:48.347566 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.347543 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd11744a_1747_4d79_861b_4197fa608a0b.slice/crio-276d388f70baae174ad2395a866a6aaeebce6ca3659c8eadb6676c9f16b631ac WatchSource:0}: Error finding container 276d388f70baae174ad2395a866a6aaeebce6ca3659c8eadb6676c9f16b631ac: Status 404 returned error can't find the container with id 276d388f70baae174ad2395a866a6aaeebce6ca3659c8eadb6676c9f16b631ac Jul 10 20:37:48.350511 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.350476 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f8dde8_d179_4c65_b586_58d76062a729.slice/crio-444ece1e94e6a046be56d98802e06dcf4fe32f078a4749fe2012f24b8f90dda1 WatchSource:0}: Error finding container 444ece1e94e6a046be56d98802e06dcf4fe32f078a4749fe2012f24b8f90dda1: Status 404 returned error can't find the container with id 444ece1e94e6a046be56d98802e06dcf4fe32f078a4749fe2012f24b8f90dda1 Jul 10 20:37:48.352578 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.352325 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92aabedc_2b21_4989_8acb_5179e6902cb8.slice/crio-3ffbe851025c75ea3b14bf616138a08d85c681656c803d9c310f6a35945bc76f WatchSource:0}: Error finding container 3ffbe851025c75ea3b14bf616138a08d85c681656c803d9c310f6a35945bc76f: Status 404 returned error can't find the container with id 3ffbe851025c75ea3b14bf616138a08d85c681656c803d9c310f6a35945bc76f Jul 10 20:37:48.353481 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.353300 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92929035_69df_4c3f_8e04_9d272ae67909.slice/crio-fa89e75b72bb7d6ff09fb079a0f97146ba2637fb5f822efe5126ea53299f6951 WatchSource:0}: Error finding container fa89e75b72bb7d6ff09fb079a0f97146ba2637fb5f822efe5126ea53299f6951: Status 404 returned error can't find the container with id fa89e75b72bb7d6ff09fb079a0f97146ba2637fb5f822efe5126ea53299f6951 Jul 10 20:37:48.353777 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.353610 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9d3179_349e_4c59_a906_28e4fbccffb8.slice/crio-9b69947f3336dc6004c24b86440354df62d34a51d21e1fa03442c2da01342128 WatchSource:0}: Error finding container 9b69947f3336dc6004c24b86440354df62d34a51d21e1fa03442c2da01342128: Status 404 returned error can't find the container with id 9b69947f3336dc6004c24b86440354df62d34a51d21e1fa03442c2da01342128 Jul 10 20:37:48.355987 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.355962 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f059320_52a4_4f1a_b987_0a60ed0159d7.slice/crio-de6c9c40a6f92d8d472dedb399a8fb3b64c4bfb7a5011386bf433701dbf0b8c6 WatchSource:0}: Error finding container de6c9c40a6f92d8d472dedb399a8fb3b64c4bfb7a5011386bf433701dbf0b8c6: Status 404 returned error can't find the container with id de6c9c40a6f92d8d472dedb399a8fb3b64c4bfb7a5011386bf433701dbf0b8c6 Jul 10 20:37:48.357029 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:37:48.356783 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53756faf_a958_4c95_a856_ad2593be1e1f.slice/crio-63b1cd617eac401172b07bc25d8fad771a0b48b59debaf2d652c107f7d92ed31 WatchSource:0}: Error finding container 63b1cd617eac401172b07bc25d8fad771a0b48b59debaf2d652c107f7d92ed31: Status 404 returned error can't find the container with id 63b1cd617eac401172b07bc25d8fad771a0b48b59debaf2d652c107f7d92ed31 Jul 10 20:37:48.712203 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.711988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:48.712203 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.712185 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:48.712461 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.712222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:48.712461 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.712275 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:50.712249463 +0000 UTC m=+6.060558791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:48.712461 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.712320 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:48.712461 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.712379 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:37:50.712363872 +0000 UTC m=+6.060673185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:48.814127 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:48.813437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:48.814127 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.813615 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:37:48.814127 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.813640 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:37:48.814127 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.813655 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:48.814127 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:48.813740 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:50.813697837 +0000 UTC m=+6.162007174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:49.027024 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.026975 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:32:47 +0000 UTC" deadline="2028-04-02 10:40:22.195717756 +0000 UTC" Jul 10 20:37:49.027024 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.027021 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15158h2m33.168701936s" Jul 10 20:37:49.161397 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.160862 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:49.161397 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:49.161008 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:37:49.181053 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.180918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mn55h" event={"ID":"92929035-69df-4c3f-8e04-9d272ae67909","Type":"ContainerStarted","Data":"fa89e75b72bb7d6ff09fb079a0f97146ba2637fb5f822efe5126ea53299f6951"} Jul 10 20:37:49.185500 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.185332 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" event={"ID":"92aabedc-2b21-4989-8acb-5179e6902cb8","Type":"ContainerStarted","Data":"3ffbe851025c75ea3b14bf616138a08d85c681656c803d9c310f6a35945bc76f"} Jul 10 20:37:49.190078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.189909 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" event={"ID":"a8f8dde8-d179-4c65-b586-58d76062a729","Type":"ContainerStarted","Data":"444ece1e94e6a046be56d98802e06dcf4fe32f078a4749fe2012f24b8f90dda1"} Jul 10 20:37:49.197070 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.197039 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c5g85" event={"ID":"dd11744a-1747-4d79-861b-4197fa608a0b","Type":"ContainerStarted","Data":"276d388f70baae174ad2395a866a6aaeebce6ca3659c8eadb6676c9f16b631ac"} Jul 10 20:37:49.200008 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.199940 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerStarted","Data":"63b1cd617eac401172b07bc25d8fad771a0b48b59debaf2d652c107f7d92ed31"} Jul 10 20:37:49.205635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.205603 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p9p6r" event={"ID":"71a681fe-80db-4415-acc7-c3c0417e4157","Type":"ContainerStarted","Data":"185a42a4857cd83a3cd392822b90e4741c3337a575a894e41c3f596b1c44076c"} Jul 10 20:37:49.209992 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.209957 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fhvv8" event={"ID":"c1370c75-ebdf-40e1-9e7f-4ca40c55c869","Type":"ContainerStarted","Data":"45f6359ad335784b91f82d14501dc1a602ecce2ec0348bc3b9943554e33bd1f7"} Jul 10 20:37:49.232315 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.231296 2577 generic.go:356] "Generic (PLEG): container finished" podID="13ed74dbe625057f78bd09ff505253e2" containerID="434f26d7e4348fe07ecdf8aa0a4b3df544273623bc412047d838f88661d26f59" exitCode=0 Jul 10 20:37:49.232315 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.231399 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" event={"ID":"13ed74dbe625057f78bd09ff505253e2","Type":"ContainerDied","Data":"434f26d7e4348fe07ecdf8aa0a4b3df544273623bc412047d838f88661d26f59"} Jul 10 20:37:49.244886 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.244331 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" event={"ID":"d8de27905a457ddc122bd67be75fc31e","Type":"ContainerStarted","Data":"14f5c6da6c2421f2433ad6af735900f233907e366e3d2405ac2c4e6212f73596"} Jul 10 20:37:49.265947 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.265806 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jvmm8" event={"ID":"9f059320-52a4-4f1a-b987-0a60ed0159d7","Type":"ContainerStarted","Data":"de6c9c40a6f92d8d472dedb399a8fb3b64c4bfb7a5011386bf433701dbf0b8c6"} Jul 10 20:37:49.277449 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:49.277347 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"9b69947f3336dc6004c24b86440354df62d34a51d21e1fa03442c2da01342128"} Jul 10 20:37:50.158666 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.158623 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:50.159165 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.158774 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:50.159254 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.159202 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:50.159398 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.159339 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:50.294891 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.294849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" event={"ID":"13ed74dbe625057f78bd09ff505253e2","Type":"ContainerStarted","Data":"4bea45b95cc3f31675f45e4328ea7b64e2a2d290c7025521e5969d1aa5094565"} Jul 10 20:37:50.313825 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.313763 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-104.ec2.internal" podStartSLOduration=3.313723411 podStartE2EDuration="3.313723411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:37:49.274889946 +0000 UTC m=+4.623199280" watchObservedRunningTime="2026-07-10 20:37:50.313723411 +0000 UTC m=+5.662032745" Jul 10 20:37:50.731753 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.731712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:50.731952 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.731768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:50.731952 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.731903 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:50.732070 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.731969 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:37:54.73194904 +0000 UTC m=+10.080258350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:50.732404 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.732374 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:50.732476 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.732438 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:37:54.732420959 +0000 UTC m=+10.080730290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:50.832408 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:50.832327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:50.832589 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.832527 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:37:50.832589 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.832549 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:37:50.832589 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.832561 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:50.832794 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:50.832631 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:54.832609349 +0000 UTC m=+10.180918662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:51.160915 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:51.160876 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:51.161416 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:51.161038 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:37:52.159216 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:52.158772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:52.159216 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:52.158941 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:52.159683 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:52.159517 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:52.159683 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:52.159639 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:53.158953 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:53.158922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:53.159437 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:53.159031 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:37:54.158764 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:54.158534 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:54.158764 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.158668 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:54.158764 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:54.158695 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:54.159399 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.158829 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:54.765779 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:54.765187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:54.765779 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:54.765261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:54.765779 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.765411 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:54.765779 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.765477 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:38:02.76545631 +0000 UTC m=+18.113765635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:37:54.765779 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.765649 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:54.765779 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.765727 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:38:02.765705086 +0000 UTC m=+18.114014528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:37:54.866541 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:54.866073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:54.866541 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.866318 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:37:54.866541 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.866341 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:37:54.866541 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.866354 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:54.866541 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:54.866427 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:02.866407923 +0000 UTC m=+18.214717257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:37:55.159018 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:55.158928 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:55.159396 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:55.159074 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:37:56.158375 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:56.158339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:56.158561 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:56.158461 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:56.158718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:56.158691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:56.158851 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:56.158830 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:57.164403 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:57.163984 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:57.164403 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:57.164102 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:37:58.158555 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:58.158523 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:37:58.158783 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:58.158644 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:37:58.158783 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:58.158690 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:37:58.158905 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:58.158815 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:37:59.158840 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:37:59.158808 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:37:59.159247 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:37:59.158938 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:00.158607 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:00.158530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:00.158607 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:00.158560 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:00.158819 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:00.158667 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:00.158865 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:00.158810 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:01.158807 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:01.158773 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:01.158977 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:01.158904 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:02.158531 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:02.158495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:02.158785 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:02.158495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:02.158785 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.158630 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:02.158785 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.158708 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:02.830774 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:02.830743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:02.830774 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:02.830780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:02.831266 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.830902 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:38:02.831266 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.830968 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:38:18.830946872 +0000 UTC m=+34.179256180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:38:02.831266 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.830903 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:38:02.831266 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.831043 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:38:18.831022746 +0000 UTC m=+34.179332058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:38:02.931900 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:02.931866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:02.932071 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.932025 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:38:02.932071 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.932052 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:38:02.932071 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.932066 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:38:02.932199 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:02.932130 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:18.932110969 +0000 UTC m=+34.280420280 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:38:03.159100 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:03.159014 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:03.159318 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:03.159135 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:04.158346 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:04.158314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:04.158831 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:04.158315 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:04.158831 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:04.158442 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:04.158831 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:04.158530 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:05.161818 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:05.159565 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:05.161818 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:05.159728 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:06.158474 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:06.158446 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:06.158622 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:06.158477 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:06.158622 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:06.158557 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:06.158762 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:06.158634 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:07.158640 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:07.158607 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:07.159069 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:07.158772 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:08.158628 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.158353 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:08.158761 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.158435 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:08.158761 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:08.158741 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:08.159217 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:08.158776 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:08.339194 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.339164 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"8ba13127d1362e5bf52b0161c2a3e75cc48f9334d16658e59cc9874f4df3ccca"} Jul 10 20:38:08.339298 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.339254 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"55cb929c75f6f833f0c6a3dda4998dc3ea2a71d91d20c28605b7c6a6773e04d0"} Jul 10 20:38:08.339357 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.339303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"f5e8b2081cb9155a3bed7fa221f5f9fc9807ce5fe7d5fc44bfc2ddcc1e5dba63"} Jul 10 20:38:08.339357 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.339350 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"e16b66dd1d3bfeb3f2b21071296ab313b4c35d7e2e7cdc35ba4c07b24911e776"} Jul 10 20:38:08.339464 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.339379 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"005df9a0248ecff8de09c987efc6a332033fbe6d1f2f8bb8d7ee8339f162b671"} Jul 10 20:38:08.340859 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.340826 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mn55h" event={"ID":"92929035-69df-4c3f-8e04-9d272ae67909","Type":"ContainerStarted","Data":"563e36ac3f7f893d94762491a9b1f1399e9d00e9bd3732c54ae69e9905655a9e"} Jul 10 20:38:08.342357 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.342336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" event={"ID":"92aabedc-2b21-4989-8acb-5179e6902cb8","Type":"ContainerStarted","Data":"dbfc08653ca4f0f693850e9bc882f1635fd05e840d7f46b2f4d4eeb93faa181a"} Jul 10 20:38:08.343865 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.343843 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" event={"ID":"a8f8dde8-d179-4c65-b586-58d76062a729","Type":"ContainerStarted","Data":"589c4bbd18aed1816f25ea3908ddd375202695e707b8f3b3057fbe33099291df"} Jul 10 20:38:08.345320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.345299 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c5g85" event={"ID":"dd11744a-1747-4d79-861b-4197fa608a0b","Type":"ContainerStarted","Data":"2a20932afb7276b323b404ea749cf9657ca3638c5edab4e126406555d6db619a"} Jul 10 20:38:08.346917 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.346891 2577 generic.go:356] "Generic (PLEG): container finished" podID="53756faf-a958-4c95-a856-ad2593be1e1f" containerID="914fe9293dfe6185542c2b94734f82d4a10657d2aecd8b96f8e33f56363a465c" exitCode=0 Jul 10 20:38:08.347000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.346964 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerDied","Data":"914fe9293dfe6185542c2b94734f82d4a10657d2aecd8b96f8e33f56363a465c"} Jul 10 20:38:08.348695 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.348672 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p9p6r" event={"ID":"71a681fe-80db-4415-acc7-c3c0417e4157","Type":"ContainerStarted","Data":"f9fe4692d5a2ec2380c6859f858e1ef925383bc623d2ab43bf59115418e985eb"} Jul 10 20:38:08.350272 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.350251 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jvmm8" event={"ID":"9f059320-52a4-4f1a-b987-0a60ed0159d7","Type":"ContainerStarted","Data":"ff932b3131bbd28cb2aa2a82e4e5198700f4c690bf0083d143289dfb57d02a25"} Jul 10 20:38:08.355058 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.355018 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-104.ec2.internal" podStartSLOduration=21.355003203 podStartE2EDuration="21.355003203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:37:50.314095699 +0000 UTC m=+5.662405037" watchObservedRunningTime="2026-07-10 20:38:08.355003203 +0000 UTC m=+23.703312534" Jul 10 20:38:08.370206 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.370160 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mn55h" podStartSLOduration=3.052470206 podStartE2EDuration="22.370147285s" totalImagesPullingTime="19.317677079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:08.354947546 +0000 UTC m=+23.703256876" watchObservedRunningTime="2026-07-10 20:38:08.370147285 +0000 UTC m=+23.718456615" Jul 10 20:38:08.370482 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.370440 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c5g85" podStartSLOduration=2.995249993 podStartE2EDuration="22.370432297s" totalImagesPullingTime="19.375182304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:08.370105481 +0000 UTC m=+23.718414806" watchObservedRunningTime="2026-07-10 20:38:08.370432297 +0000 UTC m=+23.718741629" Jul 10 20:38:08.386019 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.385976 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c4fmw" podStartSLOduration=3.042238516 podStartE2EDuration="22.385968692s" totalImagesPullingTime="19.343730176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:08.385777885 +0000 UTC m=+23.734087213" watchObservedRunningTime="2026-07-10 20:38:08.385968692 +0000 UTC m=+23.734278021" Jul 10 20:38:08.421812 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.421740 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jvmm8" podStartSLOduration=3.104857728 podStartE2EDuration="22.421725057s" totalImagesPullingTime="19.316867329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:08.421495967 +0000 UTC m=+23.769805298" watchObservedRunningTime="2026-07-10 20:38:08.421725057 +0000 UTC m=+23.770034388" Jul 10 20:38:08.436602 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:08.436558 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p9p6r" podStartSLOduration=3.111745241 podStartE2EDuration="22.436545919s" totalImagesPullingTime="19.324800678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:08.436521867 +0000 UTC m=+23.784831197" watchObservedRunningTime="2026-07-10 20:38:08.436545919 +0000 UTC m=+23.784855249" Jul 10 20:38:09.057434 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:09.057412 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 20:38:09.162045 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:09.162008 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:09.162531 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:09.162123 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:09.354995 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:09.354958 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fhvv8" event={"ID":"c1370c75-ebdf-40e1-9e7f-4ca40c55c869","Type":"ContainerStarted","Data":"b3a4bc6d517b22e63babb4fda9c48a2e922c2ec279dd380e2407c5378c04b0be"} Jul 10 20:38:09.358471 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:09.358443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"26d2a68dfba69a669c73b6481e1e1f08116257c9f11315a8b15c8aa9f1793195"} Jul 10 20:38:09.360471 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:09.360431 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" event={"ID":"a8f8dde8-d179-4c65-b586-58d76062a729","Type":"ContainerStarted","Data":"75acf9cbae2ee9f8b1ba75da2f75a0584ee79f7d6ebd515fd161939d452f5914"} Jul 10 20:38:10.051002 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:10.050880 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T20:38:09.057430131Z","UUID":"4d5843e6-436a-4a95-9231-37c8312e842f","Handler":null,"Name":"","Endpoint":""} Jul 10 20:38:10.052581 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:10.052556 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 20:38:10.052734 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:10.052602 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 20:38:10.158806 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:10.158766 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:10.158979 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:10.158780 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:10.158979 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:10.158928 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:10.159073 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:10.159052 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:11.081747 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.081706 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:38:11.082957 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.082934 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:38:11.097951 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.097892 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fhvv8" podStartSLOduration=5.770796296 podStartE2EDuration="25.097873955s" totalImagesPullingTime="19.327077659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:09.369130829 +0000 UTC m=+24.717440160" watchObservedRunningTime="2026-07-10 20:38:11.097873955 +0000 UTC m=+26.446183286" Jul 10 20:38:11.159092 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.159063 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:11.159269 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:11.159190 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:11.369499 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.369425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"cec82b4b109a27cf46fc9db10cf656596800b6fbf453c7fb7aebb1c28125938b"} Jul 10 20:38:11.371773 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.371747 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" event={"ID":"a8f8dde8-d179-4c65-b586-58d76062a729","Type":"ContainerStarted","Data":"0eabf2b0e61740bdd803e37f8b20314d85d886f2619e357ddbc96430cbc17aec"} Jul 10 20:38:11.372087 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.372041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:38:11.372520 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.372493 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jvmm8" Jul 10 20:38:11.389575 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:11.389529 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xv8z" podStartSLOduration=3.608146221 podStartE2EDuration="25.389516236s" totalImagesPullingTime="21.781370015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:11.388726058 +0000 UTC m=+26.737035387" watchObservedRunningTime="2026-07-10 20:38:11.389516236 +0000 UTC m=+26.737825557" Jul 10 20:38:12.158799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:12.158583 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:12.159250 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:12.158646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:12.159250 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:12.158867 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:12.159250 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:12.158950 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:13.159463 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.159265 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:13.160111 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:13.159504 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:13.378966 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.378927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" event={"ID":"4c9d3179-349e-4c59-a906-28e4fbccffb8","Type":"ContainerStarted","Data":"65c68cfa1593808349291d41fa43f024c97a3e0b40f0566f290584943b8f5196"} Jul 10 20:38:13.379177 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.379154 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:38:13.380836 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.380813 2577 generic.go:356] "Generic (PLEG): container finished" podID="53756faf-a958-4c95-a856-ad2593be1e1f" containerID="a5d59770c0d2ec2d9514d0054f602a3bf4424d4aa0aa0fe53386c8e6acf5eeec" exitCode=0 Jul 10 20:38:13.380952 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.380898 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerDied","Data":"a5d59770c0d2ec2d9514d0054f602a3bf4424d4aa0aa0fe53386c8e6acf5eeec"} Jul 10 20:38:13.394792 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.394773 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:38:13.432308 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:13.432225 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" podStartSLOduration=8.043532717 podStartE2EDuration="27.432212971s" totalImagesPullingTime="19.388680254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:13.407417175 +0000 UTC m=+28.755726504" watchObservedRunningTime="2026-07-10 20:38:13.432212971 +0000 UTC m=+28.780522301" Jul 10 20:38:14.158804 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.158775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:14.159055 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.158776 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:14.159055 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:14.158878 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:14.159055 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:14.159030 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:14.383477 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.383451 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 20:38:14.384013 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.383985 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:38:14.400111 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.400087 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:38:14.632174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.632150 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqvmr"] Jul 10 20:38:14.632294 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.632236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:14.632339 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:14.632316 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:14.635797 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.635757 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nbn2w"] Jul 10 20:38:14.635927 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.635868 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:14.635989 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:14.635952 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:14.636464 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.636441 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fbz74"] Jul 10 20:38:14.636570 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:14.636528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:14.636669 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:14.636632 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:15.214465 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:15.214435 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:38:15.386750 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:15.386718 2577 generic.go:356] "Generic (PLEG): container finished" podID="53756faf-a958-4c95-a856-ad2593be1e1f" containerID="7f225acb3ff0651bd9dbf6ff8f99a15a25c67835b0d051c24e2953d452aeda9b" exitCode=0 Jul 10 20:38:15.387199 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:15.386797 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerDied","Data":"7f225acb3ff0651bd9dbf6ff8f99a15a25c67835b0d051c24e2953d452aeda9b"} Jul 10 20:38:16.158433 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:16.158400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:16.158585 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:16.158400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:16.158585 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:16.158508 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:16.158675 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:16.158611 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:16.158675 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:16.158409 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:16.158741 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:16.158718 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:17.396604 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:17.396373 2577 generic.go:356] "Generic (PLEG): container finished" podID="53756faf-a958-4c95-a856-ad2593be1e1f" containerID="533187746eda313d04e4c99e07ab165de31d588008c663a35ea2472625fc0529" exitCode=0 Jul 10 20:38:17.397018 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:17.396451 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerDied","Data":"533187746eda313d04e4c99e07ab165de31d588008c663a35ea2472625fc0529"} Jul 10 20:38:18.158486 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:18.158452 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:18.158486 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:18.158480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:18.158764 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.158589 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:18.158764 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:18.158612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:18.158851 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.158763 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:18.158908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.158876 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:18.854589 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:18.854555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:18.854589 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:18.854591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:18.855088 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.854715 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:38:18.855088 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.854772 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.854752633 +0000 UTC m=+66.203061941 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:38:18.855088 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.854715 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:38:18.855088 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.854840 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret podName:decac6e6-5885-4081-8f10-041e44c6a00e nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.854826505 +0000 UTC m=+66.203135831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret") pod "global-pull-secret-syncer-nbn2w" (UID: "decac6e6-5885-4081-8f10-041e44c6a00e") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:38:18.955247 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:18.955221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:18.955420 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.955353 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:38:18.955420 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.955372 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:38:18.955420 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.955383 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jp5qg for pod openshift-network-diagnostics/network-check-target-tqvmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:38:18.955587 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:18.955439 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg podName:107af500-a6f7-43f9-adc0-ccbb5d6197a1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.955420742 +0000 UTC m=+66.303730067 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jp5qg" (UniqueName: "kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg") pod "network-check-target-tqvmr" (UID: "107af500-a6f7-43f9-adc0-ccbb5d6197a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:38:20.158447 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:20.158407 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:20.159037 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:20.158540 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:20.159037 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:20.158545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:20.159037 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:20.158701 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nbn2w" podUID="decac6e6-5885-4081-8f10-041e44c6a00e" Jul 10 20:38:20.159037 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:20.158761 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:38:20.159037 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:20.158545 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqvmr" podUID="107af500-a6f7-43f9-adc0-ccbb5d6197a1" Jul 10 20:38:21.489367 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.489349 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-104.ec2.internal" event="NodeReady" Jul 10 20:38:21.489799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.489485 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 20:38:21.529855 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.529829 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-drs4v"] Jul 10 20:38:21.531797 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.531781 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.533448 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.533428 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w566r"] Jul 10 20:38:21.534694 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.534675 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h798g\"" Jul 10 20:38:21.534874 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.534857 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.534938 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.534897 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 20:38:21.534938 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.534899 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 20:38:21.537383 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.537318 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 20:38:21.538913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.537985 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8wsvp\"" Jul 10 20:38:21.538913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.538244 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 20:38:21.538913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.538449 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 20:38:21.542710 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.542688 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-drs4v"] Jul 10 20:38:21.546650 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.546594 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w566r"] Jul 10 20:38:21.673082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.673055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlb8k\" (UniqueName: \"kubernetes.io/projected/275d5012-3f9b-40e6-8517-d5be9077b5dd-kube-api-access-rlb8k\") pod \"ingress-canary-w566r\" (UID: \"275d5012-3f9b-40e6-8517-d5be9077b5dd\") " pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.673195 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.673096 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.673195 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.673113 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kqjl\" (UniqueName: \"kubernetes.io/projected/37aefdb2-595e-4f19-adc4-da96832e6150-kube-api-access-8kqjl\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.673195 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.673135 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/37aefdb2-595e-4f19-adc4-da96832e6150-config-volume\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.673340 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.673204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37aefdb2-595e-4f19-adc4-da96832e6150-tmp-dir\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.673340 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.673256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/275d5012-3f9b-40e6-8517-d5be9077b5dd-cert\") pod \"ingress-canary-w566r\" (UID: \"275d5012-3f9b-40e6-8517-d5be9077b5dd\") " pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.774378 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.774458 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kqjl\" (UniqueName: \"kubernetes.io/projected/37aefdb2-595e-4f19-adc4-da96832e6150-kube-api-access-8kqjl\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.774458 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774412 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/37aefdb2-595e-4f19-adc4-da96832e6150-config-volume\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.774458 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37aefdb2-595e-4f19-adc4-da96832e6150-tmp-dir\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.774617 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:21.774504 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:21.774617 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:21.774563 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:22.274546385 +0000 UTC m=+37.622855696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:21.774753 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774627 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/275d5012-3f9b-40e6-8517-d5be9077b5dd-cert\") pod \"ingress-canary-w566r\" (UID: \"275d5012-3f9b-40e6-8517-d5be9077b5dd\") " pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.774753 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlb8k\" (UniqueName: \"kubernetes.io/projected/275d5012-3f9b-40e6-8517-d5be9077b5dd-kube-api-access-rlb8k\") pod \"ingress-canary-w566r\" (UID: \"275d5012-3f9b-40e6-8517-d5be9077b5dd\") " pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.774857 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.774784 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37aefdb2-595e-4f19-adc4-da96832e6150-tmp-dir\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.775129 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.775105 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/37aefdb2-595e-4f19-adc4-da96832e6150-config-volume\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.778582 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.778566 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/275d5012-3f9b-40e6-8517-d5be9077b5dd-cert\") pod \"ingress-canary-w566r\" (UID: \"275d5012-3f9b-40e6-8517-d5be9077b5dd\") " pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.783003 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.782977 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlb8k\" (UniqueName: \"kubernetes.io/projected/275d5012-3f9b-40e6-8517-d5be9077b5dd-kube-api-access-rlb8k\") pod \"ingress-canary-w566r\" (UID: \"275d5012-3f9b-40e6-8517-d5be9077b5dd\") " pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:21.783187 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.783168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kqjl\" (UniqueName: \"kubernetes.io/projected/37aefdb2-595e-4f19-adc4-da96832e6150-kube-api-access-8kqjl\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:21.881435 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:21.881415 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w566r" Jul 10 20:38:22.004563 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.004414 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w566r"] Jul 10 20:38:22.007359 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:38:22.007333 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275d5012_3f9b_40e6_8517_d5be9077b5dd.slice/crio-2d778fc5cc660be44a013c5ff8f97a1479f21477b2afd899ab743de4caae6960 WatchSource:0}: Error finding container 2d778fc5cc660be44a013c5ff8f97a1479f21477b2afd899ab743de4caae6960: Status 404 returned error can't find the container with id 2d778fc5cc660be44a013c5ff8f97a1479f21477b2afd899ab743de4caae6960 Jul 10 20:38:22.158837 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.158748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:22.158837 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.158811 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:22.158995 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.158942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:22.161632 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.161608 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 20:38:22.161750 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.161680 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 20:38:22.161750 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.161711 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n6zkh\"" Jul 10 20:38:22.161750 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.161713 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 20:38:22.162478 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.162463 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ggkq4\"" Jul 10 20:38:22.162520 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.162464 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 20:38:22.277340 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.277319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:22.277462 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:22.277446 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:22.277508 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:22.277499 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:23.277485327 +0000 UTC m=+38.625794639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:22.411432 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.411355 2577 generic.go:356] "Generic (PLEG): container finished" podID="53756faf-a958-4c95-a856-ad2593be1e1f" containerID="8f0eddcdd3561fef1949740592bf926670fbd226e027308309c2cc14e72d8d2a" exitCode=0 Jul 10 20:38:22.411568 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.411455 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerDied","Data":"8f0eddcdd3561fef1949740592bf926670fbd226e027308309c2cc14e72d8d2a"} Jul 10 20:38:22.412794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:22.412677 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w566r" event={"ID":"275d5012-3f9b-40e6-8517-d5be9077b5dd","Type":"ContainerStarted","Data":"2d778fc5cc660be44a013c5ff8f97a1479f21477b2afd899ab743de4caae6960"} Jul 10 20:38:23.284933 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:23.284891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:23.285506 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:23.285082 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:23.285506 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:23.285159 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:25.285137927 +0000 UTC m=+40.633447258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:23.419005 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:23.418972 2577 generic.go:356] "Generic (PLEG): container finished" podID="53756faf-a958-4c95-a856-ad2593be1e1f" containerID="4c29ce56ae1a356401cba088df207519ce0b5af32cfd4ac9df7a4dcdd47c943b" exitCode=0 Jul 10 20:38:23.419146 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:23.419005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerDied","Data":"4c29ce56ae1a356401cba088df207519ce0b5af32cfd4ac9df7a4dcdd47c943b"} Jul 10 20:38:23.602387 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:23.602297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:23.685968 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:23.685937 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:24.424520 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.424284 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhchj" event={"ID":"53756faf-a958-4c95-a856-ad2593be1e1f","Type":"ContainerStarted","Data":"b3ff5bc2766159f25120cba5ed9d4714c7281345c5b9048e9db7089b76a9c89c"} Jul 10 20:38:24.425810 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.425776 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w566r" event={"ID":"275d5012-3f9b-40e6-8517-d5be9077b5dd","Type":"ContainerStarted","Data":"7c11eeb70b907848922c84c9f29dfded18a98690c9b4c7b4446e62203e349fb2"} Jul 10 20:38:24.446227 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.446186 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zhchj" podStartSLOduration=8.4619413 podStartE2EDuration="38.446173003s" totalImagesPullingTime="29.984231703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:24.444690143 +0000 UTC m=+39.792999473" watchObservedRunningTime="2026-07-10 20:38:24.446173003 +0000 UTC m=+39.794482332" Jul 10 20:38:24.462476 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.462435 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w566r" podStartSLOduration=1.417479363 podStartE2EDuration="3.462421256s" totalImagesPullingTime="2.044941893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:24.462140122 +0000 UTC m=+39.810449452" watchObservedRunningTime="2026-07-10 20:38:24.462421256 +0000 UTC m=+39.810730585" Jul 10 20:38:24.576637 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.576604 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:24.677142 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.677060 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:24.831948 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.831910 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:24.933889 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:24.933814 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:25.301102 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:25.301069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:25.301286 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:25.301240 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:25.301360 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:25.301328 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:29.301306748 +0000 UTC m=+44.649616068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:25.369525 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:25.369494 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:25.468599 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:25.468570 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:27.432460 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:27.432421 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:27.485367 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:27.485333 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:27.894007 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:27.893972 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:27.948563 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:27.948525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:29.328345 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:29.328304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:29.328769 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:29.328434 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:29.328769 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:29.328489 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:37.328475239 +0000 UTC m=+52.676784547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:31.216726 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:31.216676 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:31.314055 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:31.314030 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:37.382582 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:37.382544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:37.382994 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:37.382713 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:37.382994 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:37.382793 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:53.382774959 +0000 UTC m=+68.731084271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:45.344549 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:45.344506 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:45.396852 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:45.396793 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:46.408586 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:46.408556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zh7ps" Jul 10 20:38:46.560342 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:46.560312 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:46.609995 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:46.609960 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:47.491279 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.491245 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv"] Jul 10 20:38:47.493615 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.493597 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.495993 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.495972 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5vs95\"" Jul 10 20:38:47.496119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.495975 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 20:38:47.497164 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.497147 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 20:38:47.497262 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.497168 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 20:38:47.497262 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.497183 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 20:38:47.505076 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.505053 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv"] Jul 10 20:38:47.554859 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.554829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc445dde-e003-4d54-9ef7-5fb09923cce5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d6d66b988-qgnbv\" (UID: \"bc445dde-e003-4d54-9ef7-5fb09923cce5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.555020 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.554900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqcqh\" (UniqueName: \"kubernetes.io/projected/bc445dde-e003-4d54-9ef7-5fb09923cce5-kube-api-access-fqcqh\") pod \"managed-serviceaccount-addon-agent-d6d66b988-qgnbv\" (UID: \"bc445dde-e003-4d54-9ef7-5fb09923cce5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.656162 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.656122 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqcqh\" (UniqueName: \"kubernetes.io/projected/bc445dde-e003-4d54-9ef7-5fb09923cce5-kube-api-access-fqcqh\") pod \"managed-serviceaccount-addon-agent-d6d66b988-qgnbv\" (UID: \"bc445dde-e003-4d54-9ef7-5fb09923cce5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.656307 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.656193 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc445dde-e003-4d54-9ef7-5fb09923cce5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d6d66b988-qgnbv\" (UID: \"bc445dde-e003-4d54-9ef7-5fb09923cce5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.660275 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.660247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc445dde-e003-4d54-9ef7-5fb09923cce5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d6d66b988-qgnbv\" (UID: \"bc445dde-e003-4d54-9ef7-5fb09923cce5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.665128 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.665097 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqcqh\" (UniqueName: \"kubernetes.io/projected/bc445dde-e003-4d54-9ef7-5fb09923cce5-kube-api-access-fqcqh\") pod \"managed-serviceaccount-addon-agent-d6d66b988-qgnbv\" (UID: \"bc445dde-e003-4d54-9ef7-5fb09923cce5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.816236 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.816151 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:38:47.935309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:47.935280 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv"] Jul 10 20:38:47.938021 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:38:47.937995 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc445dde_e003_4d54_9ef7_5fb09923cce5.slice/crio-66357d52d9aece79e8ae885c4ec9eae336d9f9784dec8b0274b553725ce1b391 WatchSource:0}: Error finding container 66357d52d9aece79e8ae885c4ec9eae336d9f9784dec8b0274b553725ce1b391: Status 404 returned error can't find the container with id 66357d52d9aece79e8ae885c4ec9eae336d9f9784dec8b0274b553725ce1b391 Jul 10 20:38:48.481803 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:48.481765 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" event={"ID":"bc445dde-e003-4d54-9ef7-5fb09923cce5","Type":"ContainerStarted","Data":"66357d52d9aece79e8ae885c4ec9eae336d9f9784dec8b0274b553725ce1b391"} Jul 10 20:38:50.488436 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.488408 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" event={"ID":"bc445dde-e003-4d54-9ef7-5fb09923cce5","Type":"ContainerStarted","Data":"74f9017e13036ff7ea950702d19c3d42096a17a24283c2e1425db408f2b89992"} Jul 10 20:38:50.504549 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.504505 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" podStartSLOduration=1.087638636 podStartE2EDuration="3.504488589s" totalImagesPullingTime="2.416849953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:50.504124002 +0000 UTC m=+65.852433331" watchObservedRunningTime="2026-07-10 20:38:50.504488589 +0000 UTC m=+65.852797918" Jul 10 20:38:50.880899 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.880866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:50.880899 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.880903 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:38:50.883725 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.883706 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 20:38:50.883799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.883738 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 20:38:50.891561 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:50.891537 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 20:38:50.891629 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:50.891610 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:39:54.891590725 +0000 UTC m=+130.239900050 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : secret "metrics-daemon-secret" not found Jul 10 20:38:50.893777 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.893762 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/decac6e6-5885-4081-8f10-041e44c6a00e-original-pull-secret\") pod \"global-pull-secret-syncer-nbn2w\" (UID: \"decac6e6-5885-4081-8f10-041e44c6a00e\") " pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:50.975375 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.975338 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nbn2w" Jul 10 20:38:50.981594 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.981569 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:50.983964 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.983933 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 20:38:50.993900 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:50.993879 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 20:38:51.005629 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.005600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp5qg\" (UniqueName: \"kubernetes.io/projected/107af500-a6f7-43f9-adc0-ccbb5d6197a1-kube-api-access-jp5qg\") pod \"network-check-target-tqvmr\" (UID: \"107af500-a6f7-43f9-adc0-ccbb5d6197a1\") " pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:51.092626 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.092598 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nbn2w"] Jul 10 20:38:51.095601 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:38:51.095574 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecac6e6_5885_4081_8f10_041e44c6a00e.slice/crio-f998f9f4b08614659b072835f3d9712bd7ea54b57dcd3025b0ae17fc95dbc750 WatchSource:0}: Error finding container f998f9f4b08614659b072835f3d9712bd7ea54b57dcd3025b0ae17fc95dbc750: Status 404 returned error can't find the container with id f998f9f4b08614659b072835f3d9712bd7ea54b57dcd3025b0ae17fc95dbc750 Jul 10 20:38:51.281944 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.281909 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ggkq4\"" Jul 10 20:38:51.290277 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.290249 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:51.389346 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.389310 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:51.428970 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.428941 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqvmr"] Jul 10 20:38:51.431685 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:38:51.431639 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod107af500_a6f7_43f9_adc0_ccbb5d6197a1.slice/crio-1fd00f107d1fb49c8b15bacf4daf627696284e1a180325a017224a0c3a1165cc WatchSource:0}: Error finding container 1fd00f107d1fb49c8b15bacf4daf627696284e1a180325a017224a0c3a1165cc: Status 404 returned error can't find the container with id 1fd00f107d1fb49c8b15bacf4daf627696284e1a180325a017224a0c3a1165cc Jul 10 20:38:51.486985 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.486947 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:51.492087 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.492057 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nbn2w" event={"ID":"decac6e6-5885-4081-8f10-041e44c6a00e","Type":"ContainerStarted","Data":"f998f9f4b08614659b072835f3d9712bd7ea54b57dcd3025b0ae17fc95dbc750"} Jul 10 20:38:51.493532 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:51.493497 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqvmr" event={"ID":"107af500-a6f7-43f9-adc0-ccbb5d6197a1","Type":"ContainerStarted","Data":"1fd00f107d1fb49c8b15bacf4daf627696284e1a180325a017224a0c3a1165cc"} Jul 10 20:38:53.401907 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:53.401877 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:38:53.402426 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:53.402067 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:38:53.402426 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:38:53.402153 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:25.402131045 +0000 UTC m=+100.750440357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:38:56.507705 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:56.507672 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nbn2w" event={"ID":"decac6e6-5885-4081-8f10-041e44c6a00e","Type":"ContainerStarted","Data":"6c8a80f6d6d5dce89b3fad43bbdaefc38c5f05472804a1530046a7c53b9fc24d"} Jul 10 20:38:56.512355 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:56.512320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqvmr" event={"ID":"107af500-a6f7-43f9-adc0-ccbb5d6197a1","Type":"ContainerStarted","Data":"44d5736778e3a42ba15808f796c76669e4c269ecec32d19b6e3ff9a23e9d415d"} Jul 10 20:38:56.512503 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:56.512390 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:38:56.522924 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:56.522881 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nbn2w" podStartSLOduration=65.752479258 podStartE2EDuration="1m10.522868003s" totalImagesPullingTime="4.770388745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:56.521951118 +0000 UTC m=+71.870260449" watchObservedRunningTime="2026-07-10 20:38:56.522868003 +0000 UTC m=+71.871177376" Jul 10 20:38:56.535446 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:56.535400 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tqvmr" podStartSLOduration=66.102726383 podStartE2EDuration="1m10.535381979s" totalImagesPullingTime="4.432655596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:56.535333907 +0000 UTC m=+71.883643236" watchObservedRunningTime="2026-07-10 20:38:56.535381979 +0000 UTC m=+71.883691288" Jul 10 20:38:58.621219 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:58.621184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:58.720082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:38:58.720046 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:24.248094 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:39:24.248063 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:24.302495 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:39:24.302472 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:25.414455 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:39:25.414417 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:39:25.414861 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:39:25.414542 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:39:25.414861 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:39:25.414602 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:40:29.414588302 +0000 UTC m=+164.762897610 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:39:27.517323 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:39:27.517294 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tqvmr" Jul 10 20:39:54.913593 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:39:54.913534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:39:54.914112 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:39:54.913708 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 20:39:54.914112 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:39:54.913778 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:41:56.913761633 +0000 UTC m=+252.262070958 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : secret "metrics-daemon-secret" not found Jul 10 20:40:24.576910 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:40:24.576867 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-drs4v" podUID="37aefdb2-595e-4f19-adc4-da96832e6150" Jul 10 20:40:24.720532 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:24.720502 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-drs4v" Jul 10 20:40:25.169865 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:40:25.169828 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:40:29.449904 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:29.449850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:40:29.450358 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:40:29.449992 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:40:29.450358 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:40:29.450071 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls podName:37aefdb2-595e-4f19-adc4-da96832e6150 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:31.450056946 +0000 UTC m=+286.798366258 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls") pod "dns-default-drs4v" (UID: "37aefdb2-595e-4f19-adc4-da96832e6150") : secret "dns-default-metrics-tls" not found Jul 10 20:40:38.159186 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:38.159099 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:40:50.781557 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:50.781519 2577 generic.go:356] "Generic (PLEG): container finished" podID="bc445dde-e003-4d54-9ef7-5fb09923cce5" containerID="74f9017e13036ff7ea950702d19c3d42096a17a24283c2e1425db408f2b89992" exitCode=255 Jul 10 20:40:50.781973 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:50.781578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" event={"ID":"bc445dde-e003-4d54-9ef7-5fb09923cce5","Type":"ContainerDied","Data":"74f9017e13036ff7ea950702d19c3d42096a17a24283c2e1425db408f2b89992"} Jul 10 20:40:50.787182 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:50.787161 2577 scope.go:117] "RemoveContainer" containerID="74f9017e13036ff7ea950702d19c3d42096a17a24283c2e1425db408f2b89992" Jul 10 20:40:51.785431 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:40:51.785395 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" event={"ID":"bc445dde-e003-4d54-9ef7-5fb09923cce5","Type":"ContainerStarted","Data":"9e7bc9ac7f3d14e5ab66b8e16b3ad3602f96850e8a389686bd0eb2e4af1ea100"} Jul 10 20:41:21.365140 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:21.365104 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:54.035214 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.035179 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-644d6d9498-8ldjc"] Jul 10 20:41:54.038145 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.038129 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69db686bdd-kc8rg"] Jul 10 20:41:54.038261 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.038244 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.040855 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.040834 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.043324 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.043303 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 20:41:54.043324 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.043317 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 20:41:54.043488 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.043309 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 20:41:54.044919 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.044882 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 20:41:54.045008 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.044927 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 20:41:54.045008 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.044926 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dtnv5\"" Jul 10 20:41:54.045119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.045056 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 20:41:54.045119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.045111 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 20:41:54.045413 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.045396 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 20:41:54.045467 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.045403 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 20:41:54.049185 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.049169 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2p8ck\"" Jul 10 20:41:54.054991 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.054969 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 20:41:54.076555 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.076529 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644d6d9498-8ldjc"] Jul 10 20:41:54.087186 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.087163 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69db686bdd-kc8rg"] Jul 10 20:41:54.106668 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106630 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-ca-trust-extracted\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.106757 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-trusted-ca\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.106757 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxppp\" (UniqueName: \"kubernetes.io/projected/fbafa86c-d243-443d-bfcd-c78189d42283-kube-api-access-xxppp\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.106835 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-stats-auth\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.106835 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106785 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-image-registry-private-configuration\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.106835 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-certificates\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.106922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.106922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.106922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-installation-pull-secrets\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.106922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-default-certificate\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.107036 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106953 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htbgl\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-kube-api-access-htbgl\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.107036 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.106978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-bound-sa-token\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.107036 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.107004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.129801 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.129776 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf"] Jul 10 20:41:54.132601 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.132581 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld"] Jul 10 20:41:54.132882 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.132858 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" Jul 10 20:41:54.135007 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.134990 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-snbhj\"" Jul 10 20:41:54.135389 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.135371 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4"] Jul 10 20:41:54.135510 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.135496 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.137519 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.137501 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 20:41:54.138230 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.138208 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.138346 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.138231 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fjjq2\"" Jul 10 20:41:54.138346 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.138257 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:41:54.138346 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.138267 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 20:41:54.138509 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.138269 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 20:41:54.140509 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.140490 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-x7lc5\"" Jul 10 20:41:54.140599 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.140494 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 20:41:54.141940 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.141916 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 20:41:54.142896 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.142376 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 20:41:54.143741 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.143721 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 20:41:54.145675 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.145604 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf"] Jul 10 20:41:54.146892 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.146871 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld"] Jul 10 20:41:54.148765 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.148746 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4"] Jul 10 20:41:54.207387 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207358 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgq9h\" (UniqueName: \"kubernetes.io/projected/20ab3bfc-e03c-4209-abb0-981521f1a488-kube-api-access-bgq9h\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.207387 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-installation-pull-secrets\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.207543 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207412 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htbgl\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-kube-api-access-htbgl\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.207583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-ca-trust-extracted\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.207583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207575 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20ab3bfc-e03c-4209-abb0-981521f1a488-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.207645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-trusted-ca\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.207705 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207645 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20ab3bfc-e03c-4209-abb0-981521f1a488-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.207705 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxppp\" (UniqueName: \"kubernetes.io/projected/fbafa86c-d243-443d-bfcd-c78189d42283-kube-api-access-xxppp\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.207804 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz4cl\" (UniqueName: \"kubernetes.io/projected/1fedb3e4-fdea-41c0-a1df-a84d12224823-kube-api-access-mz4cl\") pod \"network-check-source-869dcfcdcd-m6dlf\" (UID: \"1fedb3e4-fdea-41c0-a1df-a84d12224823\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" Jul 10 20:41:54.207804 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-certificates\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.207804 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.207945 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.207945 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-default-certificate\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.207945 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207867 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e4d030b2-d55b-439f-81dd-04961ec3993b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.207945 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqmz\" (UniqueName: \"kubernetes.io/projected/e4d030b2-d55b-439f-81dd-04961ec3993b-kube-api-access-cxqmz\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.207945 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.207928 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:41:54.207945 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-ca-trust-extracted\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.207934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-bound-sa-token\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.208008 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:54.70797114 +0000 UTC m=+250.056280466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : secret "router-metrics-certs-default" not found Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.208068 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.208082 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644d6d9498-8ldjc: secret "image-registry-tls" not found Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.208121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.208141 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls podName:4ab37329-d104-4b3f-8ca4-9a056c7af8c7 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:54.708123988 +0000 UTC m=+250.056433298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls") pod "image-registry-644d6d9498-8ldjc" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7") : secret "image-registry-tls" not found Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.208191 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.208236 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:54.708219523 +0000 UTC m=+250.056528850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : configmap references non-existent config key: service-ca.crt Jul 10 20:41:54.208276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.208271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-stats-auth\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.208649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.208311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-image-registry-private-configuration\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.208649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.208365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-certificates\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.209297 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.209267 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-trusted-ca\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.210049 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.210030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-installation-pull-secrets\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.210374 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.210358 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-default-certificate\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.210490 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.210472 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-stats-auth\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.210646 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.210624 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-image-registry-private-configuration\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.215836 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.215815 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-bound-sa-token\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.216574 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.216554 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htbgl\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-kube-api-access-htbgl\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.216894 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.216874 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxppp\" (UniqueName: \"kubernetes.io/projected/fbafa86c-d243-443d-bfcd-c78189d42283-kube-api-access-xxppp\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.282116 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.282083 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:54.309190 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309122 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgq9h\" (UniqueName: \"kubernetes.io/projected/20ab3bfc-e03c-4209-abb0-981521f1a488-kube-api-access-bgq9h\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.309190 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20ab3bfc-e03c-4209-abb0-981521f1a488-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.309190 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20ab3bfc-e03c-4209-abb0-981521f1a488-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.309356 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309280 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz4cl\" (UniqueName: \"kubernetes.io/projected/1fedb3e4-fdea-41c0-a1df-a84d12224823-kube-api-access-mz4cl\") pod \"network-check-source-869dcfcdcd-m6dlf\" (UID: \"1fedb3e4-fdea-41c0-a1df-a84d12224823\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" Jul 10 20:41:54.309356 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e4d030b2-d55b-439f-81dd-04961ec3993b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.309454 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqmz\" (UniqueName: \"kubernetes.io/projected/e4d030b2-d55b-439f-81dd-04961ec3993b-kube-api-access-cxqmz\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.309454 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.309621 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.309539 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:54.309731 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.309644 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls podName:e4d030b2-d55b-439f-81dd-04961ec3993b nodeName:}" failed. No retries permitted until 2026-07-10 20:41:54.809623672 +0000 UTC m=+250.157932998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bf5b4" (UID: "e4d030b2-d55b-439f-81dd-04961ec3993b") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:54.309867 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.309849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20ab3bfc-e03c-4209-abb0-981521f1a488-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.310083 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.310064 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e4d030b2-d55b-439f-81dd-04961ec3993b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.311598 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.311579 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20ab3bfc-e03c-4209-abb0-981521f1a488-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.317940 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.317919 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgq9h\" (UniqueName: \"kubernetes.io/projected/20ab3bfc-e03c-4209-abb0-981521f1a488-kube-api-access-bgq9h\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ntld\" (UID: \"20ab3bfc-e03c-4209-abb0-981521f1a488\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.318482 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.318458 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqmz\" (UniqueName: \"kubernetes.io/projected/e4d030b2-d55b-439f-81dd-04961ec3993b-kube-api-access-cxqmz\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.318562 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.318464 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz4cl\" (UniqueName: \"kubernetes.io/projected/1fedb3e4-fdea-41c0-a1df-a84d12224823-kube-api-access-mz4cl\") pod \"network-check-source-869dcfcdcd-m6dlf\" (UID: \"1fedb3e4-fdea-41c0-a1df-a84d12224823\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" Jul 10 20:41:54.383173 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.383142 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:54.443766 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.443723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" Jul 10 20:41:54.453071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.453047 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" Jul 10 20:41:54.577278 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.577241 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf"] Jul 10 20:41:54.579534 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:41:54.579507 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fedb3e4_fdea_41c0_a1df_a84d12224823.slice/crio-d2705edb9c6e349a60cce80269d0cc8341b7311e40bc5a89c4f9cd9ecac20029 WatchSource:0}: Error finding container d2705edb9c6e349a60cce80269d0cc8341b7311e40bc5a89c4f9cd9ecac20029: Status 404 returned error can't find the container with id d2705edb9c6e349a60cce80269d0cc8341b7311e40bc5a89c4f9cd9ecac20029 Jul 10 20:41:54.592224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.592194 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld"] Jul 10 20:41:54.594826 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:41:54.594801 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ab3bfc_e03c_4209_abb0_981521f1a488.slice/crio-09b53e3326fc7c46c8a90c80a9a963b1e08580071f3cda86601ceb95c9c4cf79 WatchSource:0}: Error finding container 09b53e3326fc7c46c8a90c80a9a963b1e08580071f3cda86601ceb95c9c4cf79: Status 404 returned error can't find the container with id 09b53e3326fc7c46c8a90c80a9a963b1e08580071f3cda86601ceb95c9c4cf79 Jul 10 20:41:54.713649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.713610 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.713649 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.713648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.713699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.713763 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.713810 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.713825 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644d6d9498-8ldjc: secret "image-registry-tls" not found Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.713810 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:55.713795783 +0000 UTC m=+251.062105092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : configmap references non-existent config key: service-ca.crt Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.713865 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:55.713850623 +0000 UTC m=+251.062159931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : secret "router-metrics-certs-default" not found Jul 10 20:41:54.713908 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.713878 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls podName:4ab37329-d104-4b3f-8ca4-9a056c7af8c7 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:55.713870258 +0000 UTC m=+251.062179574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls") pod "image-registry-644d6d9498-8ldjc" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7") : secret "image-registry-tls" not found Jul 10 20:41:54.814176 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.814136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:54.814363 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.814271 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:54.814363 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:54.814333 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls podName:e4d030b2-d55b-439f-81dd-04961ec3993b nodeName:}" failed. No retries permitted until 2026-07-10 20:41:55.814319393 +0000 UTC m=+251.162628701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bf5b4" (UID: "e4d030b2-d55b-439f-81dd-04961ec3993b") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:54.934969 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.934883 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" event={"ID":"20ab3bfc-e03c-4209-abb0-981521f1a488","Type":"ContainerStarted","Data":"09b53e3326fc7c46c8a90c80a9a963b1e08580071f3cda86601ceb95c9c4cf79"} Jul 10 20:41:54.936213 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.936186 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" event={"ID":"1fedb3e4-fdea-41c0-a1df-a84d12224823","Type":"ContainerStarted","Data":"336c05b87b5de52dce9f6e9d9e8f3160b77e3728a652d6dd44b3e49d87e6f544"} Jul 10 20:41:54.936326 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.936231 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" event={"ID":"1fedb3e4-fdea-41c0-a1df-a84d12224823","Type":"ContainerStarted","Data":"d2705edb9c6e349a60cce80269d0cc8341b7311e40bc5a89c4f9cd9ecac20029"} Jul 10 20:41:54.950825 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:54.950773 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-m6dlf" podStartSLOduration=0.950759451 podStartE2EDuration="950.759451ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:54.950400925 +0000 UTC m=+250.298710255" watchObservedRunningTime="2026-07-10 20:41:54.950759451 +0000 UTC m=+250.299068781" Jul 10 20:41:55.721030 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:55.720990 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:55.721030 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:55.721031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:55.721063 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.721143 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.721197 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.721218 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644d6d9498-8ldjc: secret "image-registry-tls" not found Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.721210 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:57.721194645 +0000 UTC m=+253.069503957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : secret "router-metrics-certs-default" not found Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.721267 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:57.721256045 +0000 UTC m=+253.069565353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : configmap references non-existent config key: service-ca.crt Jul 10 20:41:55.721463 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.721282 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls podName:4ab37329-d104-4b3f-8ca4-9a056c7af8c7 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:57.721273766 +0000 UTC m=+253.069583077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls") pod "image-registry-644d6d9498-8ldjc" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7") : secret "image-registry-tls" not found Jul 10 20:41:55.822082 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:55.822036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:55.822230 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.822181 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:55.822269 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:55.822245 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls podName:e4d030b2-d55b-439f-81dd-04961ec3993b nodeName:}" failed. No retries permitted until 2026-07-10 20:41:57.822228718 +0000 UTC m=+253.170538027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bf5b4" (UID: "e4d030b2-d55b-439f-81dd-04961ec3993b") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:56.933340 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:56.933293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:41:56.933812 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:56.933440 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 20:41:56.933812 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:56.933528 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs podName:b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b nodeName:}" failed. No retries permitted until 2026-07-10 20:43:58.93350814 +0000 UTC m=+374.281817462 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs") pod "network-metrics-daemon-fbz74" (UID: "b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b") : secret "metrics-daemon-secret" not found Jul 10 20:41:57.741710 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:57.741640 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:57.741710 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:57.741715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:57.741757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.741817 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.741872 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.741891 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644d6d9498-8ldjc: secret "image-registry-tls" not found Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.741895 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:01.741879127 +0000 UTC m=+257.090188435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : secret "router-metrics-certs-default" not found Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.741934 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls podName:4ab37329-d104-4b3f-8ca4-9a056c7af8c7 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:01.741920441 +0000 UTC m=+257.090229749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls") pod "image-registry-644d6d9498-8ldjc" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7") : secret "image-registry-tls" not found Jul 10 20:41:57.741947 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.741950 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:01.741941911 +0000 UTC m=+257.090251219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : configmap references non-existent config key: service-ca.crt Jul 10 20:41:57.842107 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:57.842080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:41:57.842240 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.842222 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:57.842295 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:57.842281 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls podName:e4d030b2-d55b-439f-81dd-04961ec3993b nodeName:}" failed. No retries permitted until 2026-07-10 20:42:01.84226358 +0000 UTC m=+257.190572888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bf5b4" (UID: "e4d030b2-d55b-439f-81dd-04961ec3993b") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:41:57.946719 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:57.946684 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" event={"ID":"20ab3bfc-e03c-4209-abb0-981521f1a488","Type":"ContainerStarted","Data":"bf4fd062a11b7b7e99a2f89df572a0d78a14bdcabf019adffc26a6fec8dc54c5"} Jul 10 20:41:57.962585 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:57.962531 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" podStartSLOduration=0.804357067 podStartE2EDuration="3.96251159s" totalImagesPullingTime="3.158154523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:57.961156583 +0000 UTC m=+253.309465915" watchObservedRunningTime="2026-07-10 20:41:57.96251159 +0000 UTC m=+253.310820921" Jul 10 20:41:58.920622 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:58.920589 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:59.010354 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.010318 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:59.070138 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.070103 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xjq5x"] Jul 10 20:41:59.073339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.073320 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.077040 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.077011 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 20:41:59.077040 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.077018 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wt5hx\"" Jul 10 20:41:59.077251 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.077023 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 20:41:59.077434 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.077417 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 20:41:59.086967 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.086942 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 20:41:59.090368 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.090342 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xjq5x"] Jul 10 20:41:59.114507 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.114476 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:59.153354 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.153321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.153462 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.153367 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8201e185-c37c-4de1-a2d5-f7dd79598fa6-crio-socket\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.153500 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.153457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8201e185-c37c-4de1-a2d5-f7dd79598fa6-data-volume\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.153500 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.153491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88k2b\" (UniqueName: \"kubernetes.io/projected/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-api-access-88k2b\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.153591 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.153575 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.170162 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.170121 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:59.254261 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.254398 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.254398 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8201e185-c37c-4de1-a2d5-f7dd79598fa6-crio-socket\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.254398 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8201e185-c37c-4de1-a2d5-f7dd79598fa6-data-volume\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.254398 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.254357 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:41:59.254577 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.254399 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:41:59.254577 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.254416 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:59.754400826 +0000 UTC m=+255.102710134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : configmap "kube-rbac-proxy" not found Jul 10 20:41:59.254577 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88k2b\" (UniqueName: \"kubernetes.io/projected/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-api-access-88k2b\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.254577 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254449 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8201e185-c37c-4de1-a2d5-f7dd79598fa6-crio-socket\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.254577 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.254468 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:59.754451569 +0000 UTC m=+255.102760881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : secret "insights-runtime-extractor-tls" not found Jul 10 20:41:59.254788 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.254707 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8201e185-c37c-4de1-a2d5-f7dd79598fa6-data-volume\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.262716 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.262684 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88k2b\" (UniqueName: \"kubernetes.io/projected/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-api-access-88k2b\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.437922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.437884 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:59.493435 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.493397 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:59.757358 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.757328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.757507 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.757490 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:41:59.757543 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.757515 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:41:59.757576 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:41:59.757468 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:41:59.757576 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.757551 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:00.757533761 +0000 UTC m=+256.105843086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : secret "insights-runtime-extractor-tls" not found Jul 10 20:41:59.757645 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:41:59.757592 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:00.757581735 +0000 UTC m=+256.105891044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : configmap "kube-rbac-proxy" not found Jul 10 20:42:00.144081 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:00.144005 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:00.193251 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:00.193219 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:00.766161 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:00.766127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:00.766161 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:00.766163 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:00.766362 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:00.766250 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:42:00.766362 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:00.766268 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:42:00.766362 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:00.766314 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:02.766298881 +0000 UTC m=+258.114608188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : configmap "kube-rbac-proxy" not found Jul 10 20:42:00.766362 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:00.766329 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:02.766322119 +0000 UTC m=+258.114631427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : secret "insights-runtime-extractor-tls" not found Jul 10 20:42:00.859424 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:00.859397 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:00.960217 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:00.960191 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:01.736770 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.736736 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-6xz5b"] Jul 10 20:42:01.739915 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.739899 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.742353 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.742327 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 10 20:42:01.742482 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.742383 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 10 20:42:01.742482 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.742384 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 10 20:42:01.743399 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.743383 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 10 20:42:01.743495 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.743410 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tc8v8\"" Jul 10 20:42:01.747609 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.747589 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-6xz5b"] Jul 10 20:42:01.771296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.771270 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:01.772282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.772265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:01.772333 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.772292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:01.772333 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.772322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:01.772437 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.772401 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:42:01.772437 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.772422 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:42:01.772437 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.772435 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644d6d9498-8ldjc: secret "image-registry-tls" not found Jul 10 20:42:01.772520 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.772446 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:09.772429534 +0000 UTC m=+265.120738843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : configmap references non-existent config key: service-ca.crt Jul 10 20:42:01.772520 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.772472 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:09.772461857 +0000 UTC m=+265.120771171 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : secret "router-metrics-certs-default" not found Jul 10 20:42:01.772520 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.772486 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls podName:4ab37329-d104-4b3f-8ca4-9a056c7af8c7 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:09.77247919 +0000 UTC m=+265.120788500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls") pod "image-registry-644d6d9498-8ldjc" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7") : secret "image-registry-tls" not found Jul 10 20:42:01.873790 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.873754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:42:01.873992 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.873831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8a371c03-0f00-442f-b8a0-ef4ba6886619-signing-key\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.873992 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.873900 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:42:01.873992 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:01.873967 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls podName:e4d030b2-d55b-439f-81dd-04961ec3993b nodeName:}" failed. No retries permitted until 2026-07-10 20:42:09.873953556 +0000 UTC m=+265.222262864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bf5b4" (UID: "e4d030b2-d55b-439f-81dd-04961ec3993b") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:42:01.874157 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.873999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8a371c03-0f00-442f-b8a0-ef4ba6886619-signing-cabundle\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.874157 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.874015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp7t7\" (UniqueName: \"kubernetes.io/projected/8a371c03-0f00-442f-b8a0-ef4ba6886619-kube-api-access-tp7t7\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.975210 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.975168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8a371c03-0f00-442f-b8a0-ef4ba6886619-signing-key\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.975352 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.975281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8a371c03-0f00-442f-b8a0-ef4ba6886619-signing-cabundle\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.975352 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.975308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp7t7\" (UniqueName: \"kubernetes.io/projected/8a371c03-0f00-442f-b8a0-ef4ba6886619-kube-api-access-tp7t7\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.975954 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.975923 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8a371c03-0f00-442f-b8a0-ef4ba6886619-signing-cabundle\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.977757 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.977735 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8a371c03-0f00-442f-b8a0-ef4ba6886619-signing-key\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:01.983095 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:01.983074 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp7t7\" (UniqueName: \"kubernetes.io/projected/8a371c03-0f00-442f-b8a0-ef4ba6886619-kube-api-access-tp7t7\") pod \"service-ca-79f68b9f4f-6xz5b\" (UID: \"8a371c03-0f00-442f-b8a0-ef4ba6886619\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:02.039792 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.039767 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mn55h_92929035-69df-4c3f-8e04-9d272ae67909/dns-node-resolver/0.log" Jul 10 20:42:02.049932 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.049907 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" Jul 10 20:42:02.173035 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.173002 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-6xz5b"] Jul 10 20:42:02.176029 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:02.176005 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a371c03_0f00_442f_b8a0_ef4ba6886619.slice/crio-c4b84e0419b4246b28966c99c71a53947d6c7e45b47b7e03ccf33cd8390c01f1 WatchSource:0}: Error finding container c4b84e0419b4246b28966c99c71a53947d6c7e45b47b7e03ccf33cd8390c01f1: Status 404 returned error can't find the container with id c4b84e0419b4246b28966c99c71a53947d6c7e45b47b7e03ccf33cd8390c01f1 Jul 10 20:42:02.640148 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.640116 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p9p6r_71a681fe-80db-4415-acc7-c3c0417e4157/node-ca/0.log" Jul 10 20:42:02.665597 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.665558 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:02.781446 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.781338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:02.781928 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:02.781493 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:42:02.781928 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:02.781568 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:06.781544143 +0000 UTC m=+262.129853455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : secret "insights-runtime-extractor-tls" not found Jul 10 20:42:02.781928 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.781720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:02.781928 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:02.781801 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:42:02.781928 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:02.781847 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:06.781834089 +0000 UTC m=+262.130143411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : configmap "kube-rbac-proxy" not found Jul 10 20:42:02.964569 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:02.964465 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" event={"ID":"8a371c03-0f00-442f-b8a0-ef4ba6886619","Type":"ContainerStarted","Data":"c4b84e0419b4246b28966c99c71a53947d6c7e45b47b7e03ccf33cd8390c01f1"} Jul 10 20:42:03.840485 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:03.840445 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w566r_275d5012-3f9b-40e6-8517-d5be9077b5dd/serve-healthcheck-canary/0.log" Jul 10 20:42:04.441816 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:04.441781 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2ntld_20ab3bfc-e03c-4209-abb0-981521f1a488/kube-storage-version-migrator-operator/0.log" Jul 10 20:42:04.972136 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:04.972101 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" event={"ID":"8a371c03-0f00-442f-b8a0-ef4ba6886619","Type":"ContainerStarted","Data":"63087f7be95c323f135953ae95e4895b4e79268d11374bdc228501a7d49737cf"} Jul 10 20:42:04.987499 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:04.987453 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-6xz5b" podStartSLOduration=2.206177772 podStartE2EDuration="3.9874377s" totalImagesPullingTime="1.781259928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:04.986584363 +0000 UTC m=+260.334893700" watchObservedRunningTime="2026-07-10 20:42:04.9874377 +0000 UTC m=+260.335747030" Jul 10 20:42:05.483533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:05.483497 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:05.592677 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:05.592617 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:05.684591 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:05.684540 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:05.785955 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:05.785918 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:05.922789 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:05.922754 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:06.374856 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:06.374814 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:06.583899 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:06.583848 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:06.819394 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:06.819363 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:06.819566 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:06.819404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:06.819566 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:06.819494 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:42:06.819566 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:06.819514 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:42:06.819679 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:06.819568 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:14.819548222 +0000 UTC m=+270.167857529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : configmap "kube-rbac-proxy" not found Jul 10 20:42:06.819679 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:06.819588 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:14.819581573 +0000 UTC m=+270.167890880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : secret "insights-runtime-extractor-tls" not found Jul 10 20:42:07.582951 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:07.582919 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:07.982539 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:07.982464 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:09.014835 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.014803 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:09.072688 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.072635 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:09.366550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.366463 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:09.469117 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.469081 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:09.844876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.844833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:09.844876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.844876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:09.845133 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.844925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:09.845194 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:09.845149 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle podName:fbafa86c-d243-443d-bfcd-c78189d42283 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:25.845123054 +0000 UTC m=+281.193432362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle") pod "router-default-69db686bdd-kc8rg" (UID: "fbafa86c-d243-443d-bfcd-c78189d42283") : configmap references non-existent config key: service-ca.crt Jul 10 20:42:09.847291 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.847263 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"image-registry-644d6d9498-8ldjc\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:09.847408 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.847296 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbafa86c-d243-443d-bfcd-c78189d42283-metrics-certs\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:09.945929 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.945893 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:42:09.946101 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:09.946026 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:42:09.946101 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:09.946099 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls podName:e4d030b2-d55b-439f-81dd-04961ec3993b nodeName:}" failed. No retries permitted until 2026-07-10 20:42:25.946071994 +0000 UTC m=+281.294381302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bf5b4" (UID: "e4d030b2-d55b-439f-81dd-04961ec3993b") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:42:09.947862 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.947830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:09.974289 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:09.974258 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:10.069749 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:10.069724 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644d6d9498-8ldjc"] Jul 10 20:42:10.071948 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:10.071920 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab37329_d104_4b3f_8ca4_9a056c7af8c7.slice/crio-57b2aaa62844854c307d3620bd522bb9d046701542460a334e8c0e4a1d76fe31 WatchSource:0}: Error finding container 57b2aaa62844854c307d3620bd522bb9d046701542460a334e8c0e4a1d76fe31: Status 404 returned error can't find the container with id 57b2aaa62844854c307d3620bd522bb9d046701542460a334e8c0e4a1d76fe31 Jul 10 20:42:10.991141 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:10.991106 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" event={"ID":"4ab37329-d104-4b3f-8ca4-9a056c7af8c7","Type":"ContainerStarted","Data":"7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f"} Jul 10 20:42:10.991315 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:10.991160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" event={"ID":"4ab37329-d104-4b3f-8ca4-9a056c7af8c7","Type":"ContainerStarted","Data":"57b2aaa62844854c307d3620bd522bb9d046701542460a334e8c0e4a1d76fe31"} Jul 10 20:42:10.991315 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:10.991235 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:10.996851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:10.996811 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:11.010667 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:11.010608 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" podStartSLOduration=17.010593044 podStartE2EDuration="17.010593044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:11.009831833 +0000 UTC m=+266.358141164" watchObservedRunningTime="2026-07-10 20:42:11.010593044 +0000 UTC m=+266.358902373" Jul 10 20:42:11.058526 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:11.058499 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:11.374895 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:11.374808 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.031412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.031377 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.131708 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.131674 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.231908 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.231871 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.576404 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.576371 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.774198 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.774164 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.885091 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.884996 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:12.936361 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:12.936334 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:13.174056 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:13.173979 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:13.976510 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:13.976479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.179203 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.179147 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.300469 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.300433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.375458 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.375431 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.775048 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.775019 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.886785 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.886753 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:14.886952 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.886793 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:14.886952 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:14.886933 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:42:14.887086 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:14.886999 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls podName:8201e185-c37c-4de1-a2d5-f7dd79598fa6 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:30.88698599 +0000 UTC m=+286.235295297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xjq5x" (UID: "8201e185-c37c-4de1-a2d5-f7dd79598fa6") : secret "insights-runtime-extractor-tls" not found Jul 10 20:42:14.887428 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:14.887410 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8201e185-c37c-4de1-a2d5-f7dd79598fa6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:15.572308 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:15.572257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:15.772897 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:15.772867 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:15.998111 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:15.998085 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:16.174584 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:16.174555 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:21.628504 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:21.628471 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644d6d9498-8ldjc"] Jul 10 20:42:21.738273 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:21.738222 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:21.838956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:21.838927 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:25.865316 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:25.865283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:25.865884 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:25.865863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbafa86c-d243-443d-bfcd-c78189d42283-service-ca-bundle\") pod \"router-default-69db686bdd-kc8rg\" (UID: \"fbafa86c-d243-443d-bfcd-c78189d42283\") " pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:25.966057 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:25.966019 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:42:25.968352 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:25.968324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4d030b2-d55b-439f-81dd-04961ec3993b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bf5b4\" (UID: \"e4d030b2-d55b-439f-81dd-04961ec3993b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:42:26.153248 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:26.153164 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:26.257954 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:26.257926 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" Jul 10 20:42:26.281408 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:26.281269 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69db686bdd-kc8rg"] Jul 10 20:42:26.283669 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:26.283631 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbafa86c_d243_443d_bfcd_c78189d42283.slice/crio-3f02a63eb4373860808d4a6a618b1f70805f1a159033b570d292965c1d308d1c WatchSource:0}: Error finding container 3f02a63eb4373860808d4a6a618b1f70805f1a159033b570d292965c1d308d1c: Status 404 returned error can't find the container with id 3f02a63eb4373860808d4a6a618b1f70805f1a159033b570d292965c1d308d1c Jul 10 20:42:26.390252 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:26.389235 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4"] Jul 10 20:42:26.391774 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:26.391748 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d030b2_d55b_439f_81dd_04961ec3993b.slice/crio-218b7f86f51d662a3570555af49edff0748b7f4bda03c7dffc0c0433548ca22f WatchSource:0}: Error finding container 218b7f86f51d662a3570555af49edff0748b7f4bda03c7dffc0c0433548ca22f: Status 404 returned error can't find the container with id 218b7f86f51d662a3570555af49edff0748b7f4bda03c7dffc0c0433548ca22f Jul 10 20:42:27.040336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:27.040301 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" event={"ID":"e4d030b2-d55b-439f-81dd-04961ec3993b","Type":"ContainerStarted","Data":"218b7f86f51d662a3570555af49edff0748b7f4bda03c7dffc0c0433548ca22f"} Jul 10 20:42:27.042291 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:27.042257 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69db686bdd-kc8rg" event={"ID":"fbafa86c-d243-443d-bfcd-c78189d42283","Type":"ContainerStarted","Data":"1d46e3bea4d63c4f332975a738b23d820e919e54f665494301776d95ef80722f"} Jul 10 20:42:27.042431 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:27.042309 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69db686bdd-kc8rg" event={"ID":"fbafa86c-d243-443d-bfcd-c78189d42283","Type":"ContainerStarted","Data":"3f02a63eb4373860808d4a6a618b1f70805f1a159033b570d292965c1d308d1c"} Jul 10 20:42:27.062235 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:27.062189 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-69db686bdd-kc8rg" podStartSLOduration=33.06217413 podStartE2EDuration="33.06217413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:27.062064698 +0000 UTC m=+282.410374027" watchObservedRunningTime="2026-07-10 20:42:27.06217413 +0000 UTC m=+282.410483460" Jul 10 20:42:27.153359 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:27.153320 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:27.156397 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:27.156374 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:27.721780 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:27.721735 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-drs4v" podUID="37aefdb2-595e-4f19-adc4-da96832e6150" Jul 10 20:42:28.046951 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.046920 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" event={"ID":"e4d030b2-d55b-439f-81dd-04961ec3993b","Type":"ContainerStarted","Data":"14dc31066f2324533b9d95344a0855c978538da52e0872df560726e04ece681f"} Jul 10 20:42:28.047344 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.046952 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-drs4v" Jul 10 20:42:28.047344 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.047186 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:28.048341 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.048322 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69db686bdd-kc8rg" Jul 10 20:42:28.064578 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.064539 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bf5b4" podStartSLOduration=32.511532096 podStartE2EDuration="34.064526128s" totalImagesPullingTime="1.552994032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:28.063449274 +0000 UTC m=+283.411758604" watchObservedRunningTime="2026-07-10 20:42:28.064526128 +0000 UTC m=+283.412835457" Jul 10 20:42:28.284768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.284740 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:28.385106 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.385061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:28.458265 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.458222 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:28.461412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.461389 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj"] Jul 10 20:42:28.464372 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.464358 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:28.466889 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.466865 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 20:42:28.467004 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.466872 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jhmjc\"" Jul 10 20:42:28.471690 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.471591 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj"] Jul 10 20:42:28.484560 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.484535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/922c7a96-bf9b-4ead-904a-683097ec926a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-g98lj\" (UID: \"922c7a96-bf9b-4ead-904a-683097ec926a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:28.585421 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.585370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/922c7a96-bf9b-4ead-904a-683097ec926a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-g98lj\" (UID: \"922c7a96-bf9b-4ead-904a-683097ec926a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:28.585532 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:28.585500 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 10 20:42:28.585587 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:28.585560 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/922c7a96-bf9b-4ead-904a-683097ec926a-tls-certificates podName:922c7a96-bf9b-4ead-904a-683097ec926a nodeName:}" failed. No retries permitted until 2026-07-10 20:42:29.085546062 +0000 UTC m=+284.433855375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/922c7a96-bf9b-4ead-904a-683097ec926a-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-g98lj" (UID: "922c7a96-bf9b-4ead-904a-683097ec926a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 10 20:42:28.641953 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:28.641928 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:29.089356 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:29.089327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/922c7a96-bf9b-4ead-904a-683097ec926a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-g98lj\" (UID: \"922c7a96-bf9b-4ead-904a-683097ec926a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:29.091768 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:29.091750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/922c7a96-bf9b-4ead-904a-683097ec926a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-g98lj\" (UID: \"922c7a96-bf9b-4ead-904a-683097ec926a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:29.375382 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:29.375314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:29.503919 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:29.503886 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj"] Jul 10 20:42:29.507954 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:29.507927 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922c7a96_bf9b_4ead_904a_683097ec926a.slice/crio-bb892925a39ba54bb37b95816458a0a87cb300ccb1b7eda6c08c35fdff220926 WatchSource:0}: Error finding container bb892925a39ba54bb37b95816458a0a87cb300ccb1b7eda6c08c35fdff220926: Status 404 returned error can't find the container with id bb892925a39ba54bb37b95816458a0a87cb300ccb1b7eda6c08c35fdff220926 Jul 10 20:42:30.057335 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:30.057303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" event={"ID":"922c7a96-bf9b-4ead-904a-683097ec926a","Type":"ContainerStarted","Data":"bb892925a39ba54bb37b95816458a0a87cb300ccb1b7eda6c08c35fdff220926"} Jul 10 20:42:30.905537 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:30.905458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:30.907632 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:30.907610 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8201e185-c37c-4de1-a2d5-f7dd79598fa6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xjq5x\" (UID: \"8201e185-c37c-4de1-a2d5-f7dd79598fa6\") " pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:30.966358 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:30.966321 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:31.061464 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.061441 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" event={"ID":"922c7a96-bf9b-4ead-904a-683097ec926a","Type":"ContainerStarted","Data":"03345cdad5d4431605cce6c13617fe1a3c9eae415cd45cdf3b65c737171296b8"} Jul 10 20:42:31.061669 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.061640 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:31.065468 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.065449 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:31.065989 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.065969 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" Jul 10 20:42:31.076392 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.076351 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-g98lj" podStartSLOduration=2.035046343 podStartE2EDuration="3.076339445s" totalImagesPullingTime="1.041293102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:31.075767637 +0000 UTC m=+286.424076967" watchObservedRunningTime="2026-07-10 20:42:31.076339445 +0000 UTC m=+286.424648775" Jul 10 20:42:31.166120 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.166056 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:31.182157 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.182135 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xjq5x" Jul 10 20:42:31.299295 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.299204 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xjq5x"] Jul 10 20:42:31.301833 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:31.301784 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8201e185_c37c_4de1_a2d5_f7dd79598fa6.slice/crio-46bd3d3fb13c90136468ceceb4d61447439b1c392d6a68cb255334e4481fe334 WatchSource:0}: Error finding container 46bd3d3fb13c90136468ceceb4d61447439b1c392d6a68cb255334e4481fe334: Status 404 returned error can't find the container with id 46bd3d3fb13c90136468ceceb4d61447439b1c392d6a68cb255334e4481fe334 Jul 10 20:42:31.485631 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.485574 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:31.509807 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.509782 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:42:31.511973 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.511958 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37aefdb2-595e-4f19-adc4-da96832e6150-metrics-tls\") pod \"dns-default-drs4v\" (UID: \"37aefdb2-595e-4f19-adc4-da96832e6150\") " pod="openshift-dns/dns-default-drs4v" Jul 10 20:42:31.549696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.549670 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:31.618968 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.618938 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:31.634784 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.634758 2577 patch_prober.go:28] interesting pod/image-registry-644d6d9498-8ldjc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 20:42:31.634899 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.634814 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 20:42:31.649495 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.649472 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h798g\"" Jul 10 20:42:31.657363 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.657342 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-drs4v" Jul 10 20:42:31.826414 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:31.826266 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-drs4v"] Jul 10 20:42:31.830995 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:31.830123 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37aefdb2_595e_4f19_adc4_da96832e6150.slice/crio-853ae20401265d044034011f4f6b511c1cbaf326474744c2980769a5801638d5 WatchSource:0}: Error finding container 853ae20401265d044034011f4f6b511c1cbaf326474744c2980769a5801638d5: Status 404 returned error can't find the container with id 853ae20401265d044034011f4f6b511c1cbaf326474744c2980769a5801638d5 Jul 10 20:42:32.067093 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:32.067013 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-drs4v" event={"ID":"37aefdb2-595e-4f19-adc4-da96832e6150","Type":"ContainerStarted","Data":"853ae20401265d044034011f4f6b511c1cbaf326474744c2980769a5801638d5"} Jul 10 20:42:32.069287 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:32.069258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjq5x" event={"ID":"8201e185-c37c-4de1-a2d5-f7dd79598fa6","Type":"ContainerStarted","Data":"46bd3d3fb13c90136468ceceb4d61447439b1c392d6a68cb255334e4481fe334"} Jul 10 20:42:35.700816 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:35.700749 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:36.083451 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:36.083416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-drs4v" event={"ID":"37aefdb2-595e-4f19-adc4-da96832e6150","Type":"ContainerStarted","Data":"f30fc0f099eaba1d8aa18562fac611334208331aefd60e69eb29b2264a6e8737"} Jul 10 20:42:36.083607 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:36.083478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-drs4v" event={"ID":"37aefdb2-595e-4f19-adc4-da96832e6150","Type":"ContainerStarted","Data":"573e4adb1fa6779c7abab11f8fc97a3acb7a0ebfb0361df0d53cbd7a9fe83157"} Jul 10 20:42:36.083607 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:36.083535 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-drs4v" Jul 10 20:42:36.084896 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:36.084874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjq5x" event={"ID":"8201e185-c37c-4de1-a2d5-f7dd79598fa6","Type":"ContainerStarted","Data":"5f3330cc532dd6729177a3a3fa82415a3e1fba0356431ebd753237a719a06376"} Jul 10 20:42:36.100898 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:36.100858 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-drs4v" podStartSLOduration=251.537571941 podStartE2EDuration="4m15.1008426s" totalImagesPullingTime="3.563270659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:36.100163476 +0000 UTC m=+291.448472805" watchObservedRunningTime="2026-07-10 20:42:36.1008426 +0000 UTC m=+291.449151927" Jul 10 20:42:37.093157 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.093125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjq5x" event={"ID":"8201e185-c37c-4de1-a2d5-f7dd79598fa6","Type":"ContainerStarted","Data":"8e6098f60d515e5efb07360344383971cebe4916d1fb853f3e8eeb122091a696"} Jul 10 20:42:37.641690 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.640938 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:37.875486 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.875423 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:37.914446 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.913075 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz"] Jul 10 20:42:37.918228 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.916686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:37.919704 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.919677 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 20:42:37.920858 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.920790 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-z58qd\"" Jul 10 20:42:37.920858 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.920838 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 20:42:37.921019 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.920953 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 20:42:37.931357 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.931276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz"] Jul 10 20:42:37.935479 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.935461 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wtjxf"] Jul 10 20:42:37.939207 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.939180 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:37.942613 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.941803 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-s2wjc\"" Jul 10 20:42:37.942613 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.941983 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 20:42:37.942613 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.942292 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 20:42:37.942613 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.942475 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 20:42:37.949386 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.949365 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wtjxf"] Jul 10 20:42:37.962489 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.962464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99bv8\" (UniqueName: \"kubernetes.io/projected/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-kube-api-access-99bv8\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:37.962599 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.962521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:37.962599 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.962555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:37.962736 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.962611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:37.973014 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.972994 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q7txr"] Jul 10 20:42:37.975065 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.975031 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:37.976320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.976302 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:37.978864 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.978448 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 20:42:37.978864 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.978831 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-chckk\"" Jul 10 20:42:37.979263 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.979042 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 20:42:37.979263 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:37.979209 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 20:42:38.044092 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.044058 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.063926 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.063890 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.064059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.063928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.064059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.063958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-root\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.063981 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-wtmp\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.064059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-tls\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-sys\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/db15070c-b382-462b-b6e3-db263e7227b7-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-textfile\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.064314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db15070c-b382-462b-b6e3-db263e7227b7-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.064635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mpv4\" (UniqueName: \"kubernetes.io/projected/148fe200-6d1d-4a1d-884a-92435c9726bb-kube-api-access-5mpv4\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99bv8\" (UniqueName: \"kubernetes.io/projected/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-kube-api-access-99bv8\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.064635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.064635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064428 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.064635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064453 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsbvd\" (UniqueName: \"kubernetes.io/projected/db15070c-b382-462b-b6e3-db263e7227b7-kube-api-access-jsbvd\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.064635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.064483 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/148fe200-6d1d-4a1d-884a-92435c9726bb-metrics-client-ca\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.065244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.065223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.065388 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:38.065372 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 10 20:42:38.065467 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:38.065447 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-tls podName:60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f nodeName:}" failed. No retries permitted until 2026-07-10 20:42:38.565430337 +0000 UTC m=+293.913739648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-n52fz" (UID: "60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f") : secret "openshift-state-metrics-tls" not found Jul 10 20:42:38.068717 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.068635 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.078710 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.078643 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99bv8\" (UniqueName: \"kubernetes.io/projected/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-kube-api-access-99bv8\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.164501 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.164383 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/db15070c-b382-462b-b6e3-db263e7227b7-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-textfile\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db15070c-b382-462b-b6e3-db263e7227b7-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165299 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mpv4\" (UniqueName: \"kubernetes.io/projected/148fe200-6d1d-4a1d-884a-92435c9726bb-kube-api-access-5mpv4\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsbvd\" (UniqueName: \"kubernetes.io/projected/db15070c-b382-462b-b6e3-db263e7227b7-kube-api-access-jsbvd\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/148fe200-6d1d-4a1d-884a-92435c9726bb-metrics-client-ca\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-root\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165496 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-wtmp\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165570 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-tls\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165607 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/db15070c-b382-462b-b6e3-db263e7227b7-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.166309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.165618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-sys\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.167119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.166107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/148fe200-6d1d-4a1d-884a-92435c9726bb-metrics-client-ca\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.167119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.166164 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-root\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.167119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.166219 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-wtmp\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.167119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.166367 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-textfile\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.167119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.166905 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db15070c-b382-462b-b6e3-db263e7227b7-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.167777 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.167346 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/148fe200-6d1d-4a1d-884a-92435c9726bb-sys\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.167777 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:38.166114 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 10 20:42:38.167777 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:38.167773 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-tls podName:db15070c-b382-462b-b6e3-db263e7227b7 nodeName:}" failed. No retries permitted until 2026-07-10 20:42:38.667750479 +0000 UTC m=+294.016059791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-wtjxf" (UID: "db15070c-b382-462b-b6e3-db263e7227b7") : secret "kube-state-metrics-tls" not found Jul 10 20:42:38.168353 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.168314 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.168497 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.168474 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.174233 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.174172 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.174728 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.174646 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.176386 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.175999 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/148fe200-6d1d-4a1d-884a-92435c9726bb-node-exporter-tls\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.178764 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.178724 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mpv4\" (UniqueName: \"kubernetes.io/projected/148fe200-6d1d-4a1d-884a-92435c9726bb-kube-api-access-5mpv4\") pod \"node-exporter-q7txr\" (UID: \"148fe200-6d1d-4a1d-884a-92435c9726bb\") " pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.178944 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.178888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsbvd\" (UniqueName: \"kubernetes.io/projected/db15070c-b382-462b-b6e3-db263e7227b7-kube-api-access-jsbvd\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.284361 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.284329 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.289761 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.289711 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q7txr" Jul 10 20:42:38.315046 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:38.315012 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148fe200_6d1d_4a1d_884a_92435c9726bb.slice/crio-61c066056c7fc5a80abaf5520f1d6460b99f558cfc55df0cb9d34d64a4d65165 WatchSource:0}: Error finding container 61c066056c7fc5a80abaf5520f1d6460b99f558cfc55df0cb9d34d64a4d65165: Status 404 returned error can't find the container with id 61c066056c7fc5a80abaf5520f1d6460b99f558cfc55df0cb9d34d64a4d65165 Jul 10 20:42:38.383244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.383220 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.565786 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.565757 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.569720 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.569698 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.572280 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.572259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n52fz\" (UID: \"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.670539 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.670504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.673193 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.673168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db15070c-b382-462b-b6e3-db263e7227b7-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wtjxf\" (UID: \"db15070c-b382-462b-b6e3-db263e7227b7\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.799431 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.799379 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.834427 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.834340 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" Jul 10 20:42:38.855872 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.855792 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" Jul 10 20:42:38.899760 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.899730 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:38.995979 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:38.995954 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:39.103574 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:39.103492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7txr" event={"ID":"148fe200-6d1d-4a1d-884a-92435c9726bb","Type":"ContainerStarted","Data":"61c066056c7fc5a80abaf5520f1d6460b99f558cfc55df0cb9d34d64a4d65165"} Jul 10 20:42:39.417481 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:39.417373 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:39.958356 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:39.958218 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wtjxf"] Jul 10 20:42:39.962143 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:39.961404 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb15070c_b382_462b_b6e3_db263e7227b7.slice/crio-abdee7635429b7182747177f4cc397ceb5c54214a68abac38529808aed6b2255 WatchSource:0}: Error finding container abdee7635429b7182747177f4cc397ceb5c54214a68abac38529808aed6b2255: Status 404 returned error can't find the container with id abdee7635429b7182747177f4cc397ceb5c54214a68abac38529808aed6b2255 Jul 10 20:42:39.977965 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:39.977940 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz"] Jul 10 20:42:39.984036 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:39.984007 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60fb79fb_6e7c_4eb3_b009_32cfbfd3f10f.slice/crio-2114be86d719ba8096bb15dd764e01c6d09be4409188e90231b0295271537f9b WatchSource:0}: Error finding container 2114be86d719ba8096bb15dd764e01c6d09be4409188e90231b0295271537f9b: Status 404 returned error can't find the container with id 2114be86d719ba8096bb15dd764e01c6d09be4409188e90231b0295271537f9b Jul 10 20:42:40.108989 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.108911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" event={"ID":"db15070c-b382-462b-b6e3-db263e7227b7","Type":"ContainerStarted","Data":"abdee7635429b7182747177f4cc397ceb5c54214a68abac38529808aed6b2255"} Jul 10 20:42:40.111494 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.111447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xjq5x" event={"ID":"8201e185-c37c-4de1-a2d5-f7dd79598fa6","Type":"ContainerStarted","Data":"5813accfac6268d0377dfec3effe0ef04d550c9c46fe747c2799073c58148cbe"} Jul 10 20:42:40.113211 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.113188 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" event={"ID":"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f","Type":"ContainerStarted","Data":"51cb2483367f6d4f1e9bebb6986790f10ee15838411df23e18859e5e5df5afd4"} Jul 10 20:42:40.113321 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.113233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" event={"ID":"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f","Type":"ContainerStarted","Data":"2114be86d719ba8096bb15dd764e01c6d09be4409188e90231b0295271537f9b"} Jul 10 20:42:40.128922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.128880 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xjq5x" podStartSLOduration=32.679259412 podStartE2EDuration="41.128864008s" totalImagesPullingTime="8.449604596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:41:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:40.127844117 +0000 UTC m=+295.476153470" watchObservedRunningTime="2026-07-10 20:42:40.128864008 +0000 UTC m=+295.477173337" Jul 10 20:42:40.818324 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.818291 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:40.919463 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:40.919348 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:41.120059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.119971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" event={"ID":"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f","Type":"ContainerStarted","Data":"07c421358fff89b1b3fe8e548d96943b429ee36d9a33a0c7f750126bc57b36b5"} Jul 10 20:42:41.121759 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.121730 2577 generic.go:356] "Generic (PLEG): container finished" podID="148fe200-6d1d-4a1d-884a-92435c9726bb" containerID="82bbbcdea47371a3cabee017c8c1cd16a0750618b608e662f6caa4e1ec010023" exitCode=0 Jul 10 20:42:41.121891 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.121775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7txr" event={"ID":"148fe200-6d1d-4a1d-884a-92435c9726bb","Type":"ContainerDied","Data":"82bbbcdea47371a3cabee017c8c1cd16a0750618b608e662f6caa4e1ec010023"} Jul 10 20:42:41.160689 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:41.160637 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fbz74" podUID="b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b" Jul 10 20:42:41.297174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.297145 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:41.423274 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.423174 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:41.635132 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.635100 2577 patch_prober.go:28] interesting pod/image-registry-644d6d9498-8ldjc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 20:42:41.635218 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:41.635155 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 20:42:42.127718 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.127682 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" event={"ID":"db15070c-b382-462b-b6e3-db263e7227b7","Type":"ContainerStarted","Data":"48eaa0cf26decb2b67ad48988a7022679803744ee75525fdcea9eb16b5e57150"} Jul 10 20:42:42.128136 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.127747 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" event={"ID":"db15070c-b382-462b-b6e3-db263e7227b7","Type":"ContainerStarted","Data":"c86665fa57636d1c643dea4527e6346754fe56f58b596c58d59133e1b3ddd967"} Jul 10 20:42:42.128136 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.127772 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" event={"ID":"db15070c-b382-462b-b6e3-db263e7227b7","Type":"ContainerStarted","Data":"64bb6615bcd6bd32d2fe409dc14018ea6904e50870bf7868d5400d209d347beb"} Jul 10 20:42:42.129526 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.129501 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" event={"ID":"60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f","Type":"ContainerStarted","Data":"ffdf4a3fabae644f307013ac402e1d418cd6f05d29c79bb5175e25b22d2678de"} Jul 10 20:42:42.131307 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.131284 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7txr" event={"ID":"148fe200-6d1d-4a1d-884a-92435c9726bb","Type":"ContainerStarted","Data":"d2566827f5629519b4671abb1e052e07063ac201537895975242af47ffac85f5"} Jul 10 20:42:42.131401 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.131319 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7txr" event={"ID":"148fe200-6d1d-4a1d-884a-92435c9726bb","Type":"ContainerStarted","Data":"713480f5aa3073e90b20b2184780adfe27e3c0de39c79202bc0202ccf140049c"} Jul 10 20:42:42.146435 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.146391 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-wtjxf" podStartSLOduration=3.553155632 podStartE2EDuration="5.146379803s" totalImagesPullingTime="1.593224171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:42.145581501 +0000 UTC m=+297.493890832" watchObservedRunningTime="2026-07-10 20:42:42.146379803 +0000 UTC m=+297.494689115" Jul 10 20:42:42.162541 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.162499 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n52fz" podStartSLOduration=3.740372712 podStartE2EDuration="5.162481664s" totalImagesPullingTime="1.422108952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:42.162151574 +0000 UTC m=+297.510460905" watchObservedRunningTime="2026-07-10 20:42:42.162481664 +0000 UTC m=+297.510790994" Jul 10 20:42:42.182258 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.182217 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q7txr" podStartSLOduration=3.055716104 podStartE2EDuration="5.182203863s" totalImagesPullingTime="2.063614534s" totalInitContainerRuntime="62.873225ms" isStatefulPod=true podCreationTimestamp="2026-07-10 20:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:42.181270437 +0000 UTC m=+297.529579767" watchObservedRunningTime="2026-07-10 20:42:42.182203863 +0000 UTC m=+297.530513193" Jul 10 20:42:42.227703 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.227672 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fcf7d76f4-blhvm"] Jul 10 20:42:42.233706 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.233682 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.236448 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.236418 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 20:42:42.236551 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.236530 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 20:42:42.236551 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.236536 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-295o5nuv99gk0\"" Jul 10 20:42:42.236701 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.236554 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 20:42:42.236701 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.236612 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 20:42:42.236867 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.236852 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4rzf6\"" Jul 10 20:42:42.240331 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.240301 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fcf7d76f4-blhvm"] Jul 10 20:42:42.307086 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307063 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f9d9ea2-2176-4139-9324-2e04b956bb5a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.307086 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-secret-metrics-server-client-certs\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.307262 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307123 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-secret-metrics-server-tls\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.307309 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1f9d9ea2-2176-4139-9324-2e04b956bb5a-audit-log\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.307349 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1f9d9ea2-2176-4139-9324-2e04b956bb5a-metrics-server-audit-profiles\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.307349 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307334 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-client-ca-bundle\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.307440 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.307378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p7ls\" (UniqueName: \"kubernetes.io/projected/1f9d9ea2-2176-4139-9324-2e04b956bb5a-kube-api-access-9p7ls\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408351 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-secret-metrics-server-tls\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408351 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1f9d9ea2-2176-4139-9324-2e04b956bb5a-audit-log\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408351 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1f9d9ea2-2176-4139-9324-2e04b956bb5a-metrics-server-audit-profiles\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408540 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-client-ca-bundle\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408540 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p7ls\" (UniqueName: \"kubernetes.io/projected/1f9d9ea2-2176-4139-9324-2e04b956bb5a-kube-api-access-9p7ls\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408540 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f9d9ea2-2176-4139-9324-2e04b956bb5a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408540 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408491 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-secret-metrics-server-client-certs\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.408752 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.408726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1f9d9ea2-2176-4139-9324-2e04b956bb5a-audit-log\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.409235 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.409210 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f9d9ea2-2176-4139-9324-2e04b956bb5a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.409368 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.409353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1f9d9ea2-2176-4139-9324-2e04b956bb5a-metrics-server-audit-profiles\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.410730 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.410708 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-secret-metrics-server-tls\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.411293 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.411268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-secret-metrics-server-client-certs\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.411365 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.411334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f9d9ea2-2176-4139-9324-2e04b956bb5a-client-ca-bundle\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.416414 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.416396 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p7ls\" (UniqueName: \"kubernetes.io/projected/1f9d9ea2-2176-4139-9324-2e04b956bb5a-kube-api-access-9p7ls\") pod \"metrics-server-6fcf7d76f4-blhvm\" (UID: \"1f9d9ea2-2176-4139-9324-2e04b956bb5a\") " pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.543798 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.543768 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:42:42.676992 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.676911 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:42.677899 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:42.677822 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fcf7d76f4-blhvm"] Jul 10 20:42:42.679951 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:42.679923 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9d9ea2_2176_4139_9324_2e04b956bb5a.slice/crio-e89ee6cd1294f4fdb2097a6869e97e020b85daabaadce355eb16f401dbb09f6e WatchSource:0}: Error finding container e89ee6cd1294f4fdb2097a6869e97e020b85daabaadce355eb16f401dbb09f6e: Status 404 returned error can't find the container with id e89ee6cd1294f4fdb2097a6869e97e020b85daabaadce355eb16f401dbb09f6e Jul 10 20:42:43.135607 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:43.135572 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" event={"ID":"1f9d9ea2-2176-4139-9324-2e04b956bb5a","Type":"ContainerStarted","Data":"e89ee6cd1294f4fdb2097a6869e97e020b85daabaadce355eb16f401dbb09f6e"} Jul 10 20:42:43.201953 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:43.201927 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:43.962905 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:43.962872 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:44.037010 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.036977 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:44.128157 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.126140 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:42:44.140410 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.130706 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.145875 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.145232 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 20:42:44.145875 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.145460 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 20:42:44.145875 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.145482 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-87ar61ms8vjhl\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.146181 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.146285 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.146307 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.146493 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.147684 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.147853 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.149094 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.149274 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.149469 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hwgvd\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.149636 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.150078 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.150418 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 20:42:44.150851 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.150726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 20:42:44.155673 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.153355 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:42:44.158736 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.157378 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 20:42:44.226923 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.226896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.226932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.226950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.226975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.226994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227036 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-web-config\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227224 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227184 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-config\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vxq7\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-kube-api-access-8vxq7\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227252 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-config-out\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227395 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.227529 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.227413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.240892 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.240871 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:44.328548 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328518 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328627 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328649 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328716 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-web-config\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-config\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vxq7\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-kube-api-access-8vxq7\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.328956 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328937 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.328986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-config-out\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329122 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329643 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329441 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.329838 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.329813 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.330171 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.330148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.331026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.331000 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.331797 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.331480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.331797 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.331552 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.332610 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.332581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.332869 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.332822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.333140 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.333101 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.333522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.333499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.334144 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.334107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.334249 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.334215 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-config\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.334580 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.334556 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.334675 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.334597 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.334944 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.334925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-config-out\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.335116 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.335093 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.335339 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.335320 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-web-config\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.336204 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.336180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.336838 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.336822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vxq7\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-kube-api-access-8vxq7\") pod \"prometheus-k8s-0\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.382315 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.382293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:44.462490 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.462467 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:44.594875 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:44.594853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:42:44.597695 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:42:44.597668 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231581db_c62c_4489_8087_16deddd47d73.slice/crio-81d206df36333873e369f5504af0893ae18c30250fdc09c80adf9b10ea4e4805 WatchSource:0}: Error finding container 81d206df36333873e369f5504af0893ae18c30250fdc09c80adf9b10ea4e4805: Status 404 returned error can't find the container with id 81d206df36333873e369f5504af0893ae18c30250fdc09c80adf9b10ea4e4805 Jul 10 20:42:45.035725 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:45.035683 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 20:42:45.167395 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:45.167123 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" event={"ID":"1f9d9ea2-2176-4139-9324-2e04b956bb5a","Type":"ContainerStarted","Data":"70d94c236db4ee1a2b82110e947ee34412040fb2377b29ba1725456dce7750ce"} Jul 10 20:42:45.191026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:45.168443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"81d206df36333873e369f5504af0893ae18c30250fdc09c80adf9b10ea4e4805"} Jul 10 20:42:45.191026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:45.184153 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" podStartSLOduration=1.649814927 podStartE2EDuration="3.18413537s" totalImagesPullingTime="1.534320443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:45.182315651 +0000 UTC m=+300.530624980" watchObservedRunningTime="2026-07-10 20:42:45.18413537 +0000 UTC m=+300.532444700" Jul 10 20:42:46.095572 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:46.095540 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-drs4v" Jul 10 20:42:46.173316 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:46.173286 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1" exitCode=0 Jul 10 20:42:46.173784 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:46.173371 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1"} Jul 10 20:42:46.648014 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:46.647957 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerName="registry" containerID="cri-o://7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f" gracePeriod=30 Jul 10 20:42:46.898159 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:46.898099 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:47.049642 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049612 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.049829 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049673 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-image-registry-private-configuration\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.049829 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049698 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-trusted-ca\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.049829 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049725 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-bound-sa-token\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.049829 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049755 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-installation-pull-secrets\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.049829 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049819 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-certificates\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.050095 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049851 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-ca-trust-extracted\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.050095 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.049886 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htbgl\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-kube-api-access-htbgl\") pod \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\" (UID: \"4ab37329-d104-4b3f-8ca4-9a056c7af8c7\") " Jul 10 20:42:47.050284 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.050197 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:42:47.050631 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.050571 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:42:47.052012 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.051984 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:42:47.052228 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.052203 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:42:47.052301 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.052228 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:42:47.052301 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.052208 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-kube-api-access-htbgl" (OuterVolumeSpecName: "kube-api-access-htbgl") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "kube-api-access-htbgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:42:47.052301 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.052246 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:42:47.061467 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.061438 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4ab37329-d104-4b3f-8ca4-9a056c7af8c7" (UID: "4ab37329-d104-4b3f-8ca4-9a056c7af8c7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:42:47.150876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150813 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-certificates\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.150876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150840 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-ca-trust-extracted\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.150876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150849 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htbgl\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-kube-api-access-htbgl\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.150876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150858 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-registry-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.150876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150867 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-image-registry-private-configuration\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.150876 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150876 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-trusted-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.151135 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150884 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-bound-sa-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.151135 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.150892 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ab37329-d104-4b3f-8ca4-9a056c7af8c7-installation-pull-secrets\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:42:47.178297 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.178267 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerID="7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f" exitCode=0 Jul 10 20:42:47.178605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.178310 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" Jul 10 20:42:47.178605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.178312 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" event={"ID":"4ab37329-d104-4b3f-8ca4-9a056c7af8c7","Type":"ContainerDied","Data":"7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f"} Jul 10 20:42:47.178605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.178356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d6d9498-8ldjc" event={"ID":"4ab37329-d104-4b3f-8ca4-9a056c7af8c7","Type":"ContainerDied","Data":"57b2aaa62844854c307d3620bd522bb9d046701542460a334e8c0e4a1d76fe31"} Jul 10 20:42:47.178605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.178381 2577 scope.go:117] "RemoveContainer" containerID="7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f" Jul 10 20:42:47.186573 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.186554 2577 scope.go:117] "RemoveContainer" containerID="7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f" Jul 10 20:42:47.186845 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:47.186821 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f\": container with ID starting with 7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f not found: ID does not exist" containerID="7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f" Jul 10 20:42:47.186919 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.186857 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f"} err="failed to get container status \"7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f\": rpc error: code = NotFound desc = could not find container \"7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f\": container with ID starting with 7b02d0af0979c99c9270861ef0aa41ba09f5e52331d53630e1040c49dfbd2e1f not found: ID does not exist" Jul 10 20:42:47.194190 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.194168 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644d6d9498-8ldjc"] Jul 10 20:42:47.197595 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:47.197576 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-644d6d9498-8ldjc"] Jul 10 20:42:49.165067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:49.165029 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" path="/var/lib/kubelet/pods/4ab37329-d104-4b3f-8ca4-9a056c7af8c7/volumes" Jul 10 20:42:49.880018 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:49.879996 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:42:50.152715 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:50.152616 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:50.192181 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:50.192148 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38"} Jul 10 20:42:50.192534 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:50.192208 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761"} Jul 10 20:42:51.198091 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:51.198020 2577 generic.go:356] "Generic (PLEG): container finished" podID="bc445dde-e003-4d54-9ef7-5fb09923cce5" containerID="9e7bc9ac7f3d14e5ab66b8e16b3ad3602f96850e8a389686bd0eb2e4af1ea100" exitCode=255 Jul 10 20:42:51.198526 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:51.198087 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" event={"ID":"bc445dde-e003-4d54-9ef7-5fb09923cce5","Type":"ContainerDied","Data":"9e7bc9ac7f3d14e5ab66b8e16b3ad3602f96850e8a389686bd0eb2e4af1ea100"} Jul 10 20:42:51.198526 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:51.198145 2577 scope.go:117] "RemoveContainer" containerID="74f9017e13036ff7ea950702d19c3d42096a17a24283c2e1425db408f2b89992" Jul 10 20:42:51.205489 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:51.205468 2577 scope.go:117] "RemoveContainer" containerID="9e7bc9ac7f3d14e5ab66b8e16b3ad3602f96850e8a389686bd0eb2e4af1ea100" Jul 10 20:42:51.205700 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:51.205678 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-d6d66b988-qgnbv_open-cluster-management-agent-addon(bc445dde-e003-4d54-9ef7-5fb09923cce5)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" podUID="bc445dde-e003-4d54-9ef7-5fb09923cce5" Jul 10 20:42:52.206208 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:52.206125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442"} Jul 10 20:42:52.206208 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:52.206202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac"} Jul 10 20:42:52.206577 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:52.206244 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291"} Jul 10 20:42:52.206577 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:52.206282 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerStarted","Data":"242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed"} Jul 10 20:42:52.237322 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:52.237255 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:52.237883 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:52.237844 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.8672805970000002 podStartE2EDuration="8.237828837s" totalImagesPullingTime="6.351561269s" totalInitContainerRuntime="18.986971ms" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:52.235703812 +0000 UTC m=+307.584013141" watchObservedRunningTime="2026-07-10 20:42:52.237828837 +0000 UTC m=+307.586138167" Jul 10 20:42:53.386635 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:53.386592 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:53.486668 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:53.486617 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:53.542867 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:53.542833 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:53.678408 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:53.678322 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:54.062379 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.062345 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:54.162959 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.162929 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:54.463546 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.463457 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:42:54.532906 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.532873 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:54.590158 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.590129 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:54.681799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.681770 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:54.729805 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:54.729736 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:55.077135 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:55.077102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:55.178974 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:55.178947 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:55.354382 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:55.354299 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:55.455408 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:55.455379 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:56.158817 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:56.158787 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:42:56.689479 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:56.689445 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:56.743407 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:56.743377 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:57.816473 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:57.816425 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" Jul 10 20:42:57.816903 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:57.816798 2577 scope.go:117] "RemoveContainer" containerID="9e7bc9ac7f3d14e5ab66b8e16b3ad3602f96850e8a389686bd0eb2e4af1ea100" Jul 10 20:42:57.816981 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:42:57.816960 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-d6d66b988-qgnbv_open-cluster-management-agent-addon(bc445dde-e003-4d54-9ef7-5fb09923cce5)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" podUID="bc445dde-e003-4d54-9ef7-5fb09923cce5" Jul 10 20:42:57.895153 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:57.895119 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:57.952595 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:57.952560 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:58.128390 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:58.128304 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:58.231357 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:42:58.231326 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:00.004543 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:00.004480 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:00.105174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:00.105136 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:02.544515 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:02.544482 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:43:02.544897 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:02.544539 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:43:04.190894 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:04.190856 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.259751 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:04.259717 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:09.512640 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:09.512606 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:09.570209 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:09.570172 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:12.159269 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:12.159229 2577 scope.go:117] "RemoveContainer" containerID="9e7bc9ac7f3d14e5ab66b8e16b3ad3602f96850e8a389686bd0eb2e4af1ea100" Jul 10 20:43:13.291169 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:13.291135 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d6d66b988-qgnbv" event={"ID":"bc445dde-e003-4d54-9ef7-5fb09923cce5","Type":"ContainerStarted","Data":"f7304b03da5495d35285d7702bb6b272486a09331792e67fad4f21a68e2bf585"} Jul 10 20:43:13.316545 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:13.316514 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:13.417810 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:13.417779 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:18.696167 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:18.696089 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:18.750075 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:18.750047 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:19.314724 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:19.314688 2577 generic.go:356] "Generic (PLEG): container finished" podID="20ab3bfc-e03c-4209-abb0-981521f1a488" containerID="bf4fd062a11b7b7e99a2f89df572a0d78a14bdcabf019adffc26a6fec8dc54c5" exitCode=0 Jul 10 20:43:19.314917 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:19.314753 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" event={"ID":"20ab3bfc-e03c-4209-abb0-981521f1a488","Type":"ContainerDied","Data":"bf4fd062a11b7b7e99a2f89df572a0d78a14bdcabf019adffc26a6fec8dc54c5"} Jul 10 20:43:19.315050 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:19.315037 2577 scope.go:117] "RemoveContainer" containerID="bf4fd062a11b7b7e99a2f89df572a0d78a14bdcabf019adffc26a6fec8dc54c5" Jul 10 20:43:20.319736 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:20.319702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ntld" event={"ID":"20ab3bfc-e03c-4209-abb0-981521f1a488","Type":"ContainerStarted","Data":"641f70aef1dffaecc3e94f4e7657dd164fdf2092e73f114b8572ad5d60bbde15"} Jul 10 20:43:21.298440 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:21.298410 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:21.350548 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:21.350516 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:22.287502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:22.287466 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:22.342378 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:22.342350 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:22.549675 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:22.549598 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:43:22.553522 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:22.553498 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fcf7d76f4-blhvm" Jul 10 20:43:24.298598 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:24.298565 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:27.893155 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:27.893121 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:27.942320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:27.942291 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:42.889973 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:42.889867 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:42.941594 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:42.941509 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:44.463327 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:44.463297 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:43:44.478525 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:44.478496 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:43:45.429896 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:45.429824 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:43:53.979296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:53.979263 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:54.028304 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:54.028273 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:58.974744 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:58.974696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:43:58.977133 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:58.977112 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b-metrics-certs\") pod \"network-metrics-daemon-fbz74\" (UID: \"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b\") " pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:43:59.161291 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:59.161259 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n6zkh\"" Jul 10 20:43:59.170211 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:59.170186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fbz74" Jul 10 20:43:59.293857 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:59.293635 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fbz74"] Jul 10 20:43:59.295866 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:43:59.295839 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb577ecb4_73e4_4cd5_b1a3_b97203f6fe0b.slice/crio-f01ad5d4dc87776cdfdc427ff594c35c858353a764a0783498d6cb192786886e WatchSource:0}: Error finding container f01ad5d4dc87776cdfdc427ff594c35c858353a764a0783498d6cb192786886e: Status 404 returned error can't find the container with id f01ad5d4dc87776cdfdc427ff594c35c858353a764a0783498d6cb192786886e Jul 10 20:43:59.414297 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:59.414246 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:59.473628 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:59.473595 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fbz74" event={"ID":"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b","Type":"ContainerStarted","Data":"f01ad5d4dc87776cdfdc427ff594c35c858353a764a0783498d6cb192786886e"} Jul 10 20:43:59.477039 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:43:59.477003 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:00.479449 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:00.479421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fbz74" event={"ID":"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b","Type":"ContainerStarted","Data":"216af6102c88b59d274f0ada08c4abb4b4bfba267a4f66b88a2bbb9946577869"} Jul 10 20:44:00.479801 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:00.479471 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fbz74" event={"ID":"b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b","Type":"ContainerStarted","Data":"4fa7ad7fad97771a10e2896bb4a0cb2303f10defa42e4fd555be759024426d1c"} Jul 10 20:44:00.506000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:00.502184 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fbz74" podStartSLOduration=373.516493312 podStartE2EDuration="6m14.502158616s" totalImagesPullingTime="985.665304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:00.499589342 +0000 UTC m=+375.847898674" watchObservedRunningTime="2026-07-10 20:44:00.502158616 +0000 UTC m=+375.850467950" Jul 10 20:44:00.589050 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:00.588966 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:00.648765 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:00.648733 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:01.774830 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:01.774788 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:01.832741 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:01.832707 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:02.492602 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.492566 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:44:02.493239 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.493206 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-thanos" containerID="cri-o://44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442" gracePeriod=600 Jul 10 20:44:02.493426 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.493398 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy" containerID="cri-o://d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac" gracePeriod=600 Jul 10 20:44:02.493542 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.493334 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="prometheus" containerID="cri-o://f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761" gracePeriod=600 Jul 10 20:44:02.493610 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.493425 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="config-reloader" containerID="cri-o://a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38" gracePeriod=600 Jul 10 20:44:02.493610 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.493393 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-web" containerID="cri-o://2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291" gracePeriod=600 Jul 10 20:44:02.493610 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:02.493430 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="thanos-sidecar" containerID="cri-o://242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed" gracePeriod=600 Jul 10 20:44:03.493550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493517 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442" exitCode=0 Jul 10 20:44:03.493550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493544 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac" exitCode=0 Jul 10 20:44:03.493550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493550 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed" exitCode=0 Jul 10 20:44:03.493550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493559 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38" exitCode=0 Jul 10 20:44:03.493550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493567 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761" exitCode=0 Jul 10 20:44:03.494071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493589 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442"} Jul 10 20:44:03.494071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493674 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac"} Jul 10 20:44:03.494071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493705 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed"} Jul 10 20:44:03.494071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493732 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38"} Jul 10 20:44:03.494071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.493758 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761"} Jul 10 20:44:03.734374 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.734348 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:03.814013 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.813937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814013 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.813978 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-1\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814013 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814008 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-metrics-client-certs\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814028 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-tls\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814119 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-thanos-prometheus-http-client-file\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814167 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-kubelet-serving-ca-bundle\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814196 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-metrics-client-ca\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814219 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-0\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814282 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814262 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-serving-certs-ca-bundle\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814292 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-tls-assets\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814298 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814327 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-2\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814352 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-trusted-ca-bundle\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814385 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-config\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814424 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-config-out\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814449 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-kube-rbac-proxy\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814502 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-web-config\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814860 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814538 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vxq7\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-kube-api-access-8vxq7\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814860 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814567 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814860 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814600 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-grpc-tls\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814860 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814628 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-db\") pod \"231581db-c62c-4489-8087-16deddd47d73\" (UID: \"231581db-c62c-4489-8087-16deddd47d73\") " Jul 10 20:44:03.814860 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814695 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.815107 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814917 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.815107 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.814938 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.815107 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.815019 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.815446 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.815251 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.815605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.815543 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.816103 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.815860 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.816449 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.816419 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:44:03.818150 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.818114 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:44:03.818247 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.818121 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.818247 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.818164 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.818584 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.818556 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.818726 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.818580 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:44:03.819020 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.818989 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.819114 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.819066 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-config-out" (OuterVolumeSpecName: "config-out") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:44:03.819439 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.819414 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.819519 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.819436 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-config" (OuterVolumeSpecName: "config") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.819700 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.819678 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-kube-api-access-8vxq7" (OuterVolumeSpecName: "kube-api-access-8vxq7") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "kube-api-access-8vxq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:44:03.819989 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.819973 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.820362 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.820347 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.829419 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.829398 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-web-config" (OuterVolumeSpecName: "web-config") pod "231581db-c62c-4489-8087-16deddd47d73" (UID: "231581db-c62c-4489-8087-16deddd47d73"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:44:03.915502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915463 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915494 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915503 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915512 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-config-out\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915521 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-kube-rbac-proxy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915529 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-web-config\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915537 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vxq7\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-kube-api-access-8vxq7\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915547 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915555 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-grpc-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915563 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-db\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915571 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915581 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-metrics-client-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915591 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915599 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/231581db-c62c-4489-8087-16deddd47d73-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915608 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-metrics-client-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915616 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915624 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/231581db-c62c-4489-8087-16deddd47d73-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:03.915794 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:03.915632 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/231581db-c62c-4489-8087-16deddd47d73-tls-assets\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 20:44:04.501036 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.501003 2577 generic.go:356] "Generic (PLEG): container finished" podID="231581db-c62c-4489-8087-16deddd47d73" containerID="2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291" exitCode=0 Jul 10 20:44:04.501473 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.501090 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.501473 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.501089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291"} Jul 10 20:44:04.501473 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.501210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"231581db-c62c-4489-8087-16deddd47d73","Type":"ContainerDied","Data":"81d206df36333873e369f5504af0893ae18c30250fdc09c80adf9b10ea4e4805"} Jul 10 20:44:04.501473 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.501254 2577 scope.go:117] "RemoveContainer" containerID="44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442" Jul 10 20:44:04.509688 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.509672 2577 scope.go:117] "RemoveContainer" containerID="d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac" Jul 10 20:44:04.516437 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.516422 2577 scope.go:117] "RemoveContainer" containerID="2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291" Jul 10 20:44:04.524078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.524057 2577 scope.go:117] "RemoveContainer" containerID="242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed" Jul 10 20:44:04.524600 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.524580 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:44:04.528494 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.528474 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:44:04.531149 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.531136 2577 scope.go:117] "RemoveContainer" containerID="a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38" Jul 10 20:44:04.537434 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.537420 2577 scope.go:117] "RemoveContainer" containerID="f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761" Jul 10 20:44:04.544453 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.544435 2577 scope.go:117] "RemoveContainer" containerID="b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1" Jul 10 20:44:04.551059 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.551039 2577 scope.go:117] "RemoveContainer" containerID="44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442" Jul 10 20:44:04.551402 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.551372 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442\": container with ID starting with 44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442 not found: ID does not exist" containerID="44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442" Jul 10 20:44:04.551502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.551409 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442"} err="failed to get container status \"44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442\": rpc error: code = NotFound desc = could not find container \"44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442\": container with ID starting with 44803261f30f1584f393106333ba912ccc846b91cbc30bfea63de23a207de442 not found: ID does not exist" Jul 10 20:44:04.551502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.551428 2577 scope.go:117] "RemoveContainer" containerID="d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac" Jul 10 20:44:04.551745 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.551725 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac\": container with ID starting with d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac not found: ID does not exist" containerID="d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac" Jul 10 20:44:04.551799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.551752 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac"} err="failed to get container status \"d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac\": rpc error: code = NotFound desc = could not find container \"d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac\": container with ID starting with d80469999b7a469d45a2ccd4534142c9015136a44b45d9a66149ecbfc67207ac not found: ID does not exist" Jul 10 20:44:04.551799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.551769 2577 scope.go:117] "RemoveContainer" containerID="2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291" Jul 10 20:44:04.551980 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.551959 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291\": container with ID starting with 2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291 not found: ID does not exist" containerID="2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291" Jul 10 20:44:04.552027 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.551986 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291"} err="failed to get container status \"2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291\": rpc error: code = NotFound desc = could not find container \"2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291\": container with ID starting with 2dbbe57c0e73e6774780c01e088300cdac83446faf4113db7d41f39e5ce38291 not found: ID does not exist" Jul 10 20:44:04.552027 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552000 2577 scope.go:117] "RemoveContainer" containerID="242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed" Jul 10 20:44:04.552219 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.552202 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed\": container with ID starting with 242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed not found: ID does not exist" containerID="242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed" Jul 10 20:44:04.552264 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552224 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed"} err="failed to get container status \"242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed\": rpc error: code = NotFound desc = could not find container \"242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed\": container with ID starting with 242acf222b28fe86578a4511dd05d67f71bb094743e47dac0a435edb52e931ed not found: ID does not exist" Jul 10 20:44:04.552264 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552241 2577 scope.go:117] "RemoveContainer" containerID="a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38" Jul 10 20:44:04.552464 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.552446 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38\": container with ID starting with a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38 not found: ID does not exist" containerID="a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38" Jul 10 20:44:04.552500 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552470 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38"} err="failed to get container status \"a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38\": rpc error: code = NotFound desc = could not find container \"a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38\": container with ID starting with a675199736f2c5a6fbeab9d4c6c0ea4b4cac8009521d15129464219c7b78ba38 not found: ID does not exist" Jul 10 20:44:04.552500 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552488 2577 scope.go:117] "RemoveContainer" containerID="f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761" Jul 10 20:44:04.552573 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552557 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:44:04.552757 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.552739 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761\": container with ID starting with f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761 not found: ID does not exist" containerID="f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761" Jul 10 20:44:04.552818 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552765 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761"} err="failed to get container status \"f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761\": rpc error: code = NotFound desc = could not find container \"f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761\": container with ID starting with f584d8722d2c89825ba4396bfdb723bee7952850b8dd03e69e81608f7346e761 not found: ID does not exist" Jul 10 20:44:04.552818 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552784 2577 scope.go:117] "RemoveContainer" containerID="b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1" Jul 10 20:44:04.552913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552862 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="prometheus" Jul 10 20:44:04.552913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552878 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="prometheus" Jul 10 20:44:04.552913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552891 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-web" Jul 10 20:44:04.552913 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552900 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-web" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552916 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-thanos" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552925 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-thanos" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552935 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552943 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552955 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerName="registry" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552964 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerName="registry" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552975 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="thanos-sidecar" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552983 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="thanos-sidecar" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.552995 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="init-config-reloader" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553004 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="init-config-reloader" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553014 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="config-reloader" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553019 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="config-reloader" Jul 10 20:44:04.553038 ip-10-0-136-104 kubenswrapper[2577]: E0710 20:44:04.553037 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1\": container with ID starting with b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1 not found: ID does not exist" containerID="b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553058 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1"} err="failed to get container status \"b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1\": rpc error: code = NotFound desc = could not find container \"b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1\": container with ID starting with b8d0afd67312a9fd78069db2b982ef061066cc884db3d40ec8e9bfcc3f2daad1 not found: ID does not exist" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553071 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab37329-d104-4b3f-8ca4-9a056c7af8c7" containerName="registry" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553079 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="config-reloader" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553086 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="prometheus" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553091 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-thanos" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553098 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="thanos-sidecar" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553104 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy-web" Jul 10 20:44:04.553412 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.553109 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="231581db-c62c-4489-8087-16deddd47d73" containerName="kube-rbac-proxy" Jul 10 20:44:04.558174 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.558157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.560806 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.560784 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 20:44:04.560904 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.560808 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 20:44:04.560904 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.560841 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 20:44:04.561071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561054 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 20:44:04.561141 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561111 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 20:44:04.561190 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561142 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 20:44:04.561190 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561145 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 20:44:04.561294 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561148 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hwgvd\"" Jul 10 20:44:04.561294 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561237 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 20:44:04.561390 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561306 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 20:44:04.561427 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561418 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 20:44:04.561524 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561507 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 20:44:04.561962 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.561948 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-87ar61ms8vjhl\"" Jul 10 20:44:04.562279 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.562262 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 20:44:04.564233 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.564216 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 20:44:04.570065 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.566695 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 20:44:04.573708 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.573687 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:44:04.620908 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.620883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.620919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.620939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.620958 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-config\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.620990 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a668e185-3055-4ffa-95d4-055ca09a78b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a668e185-3055-4ffa-95d4-055ca09a78b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621156 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621336 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621679 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621679 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621371 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621679 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621396 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.621679 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.621426 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf85d\" (UniqueName: \"kubernetes.io/projected/a668e185-3055-4ffa-95d4-055ca09a78b6-kube-api-access-qf85d\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.721814 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.721814 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf85d\" (UniqueName: \"kubernetes.io/projected/a668e185-3055-4ffa-95d4-055ca09a78b6-kube-api-access-qf85d\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721898 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721921 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-config\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.721977 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722032 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.722000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722688 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.722510 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722688 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.722578 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722688 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.722619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a668e185-3055-4ffa-95d4-055ca09a78b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.722931 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.722865 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a668e185-3055-4ffa-95d4-055ca09a78b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723142 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723180 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.723645 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723472 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.724206 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.723953 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.725932 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.724970 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-config\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.725932 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.725164 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.725932 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.725284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-web-config\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.725932 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.725311 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.725932 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.725817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.726987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.725982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.726987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.726091 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.726987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.726586 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a668e185-3055-4ffa-95d4-055ca09a78b6-config-out\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.726987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.726590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.726987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.726737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.726987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.726875 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.727833 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.727804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.728147 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.728126 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a668e185-3055-4ffa-95d4-055ca09a78b6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.728464 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.728443 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a668e185-3055-4ffa-95d4-055ca09a78b6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.728538 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.728449 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a668e185-3055-4ffa-95d4-055ca09a78b6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.729617 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.729592 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf85d\" (UniqueName: \"kubernetes.io/projected/a668e185-3055-4ffa-95d4-055ca09a78b6-kube-api-access-qf85d\") pod \"prometheus-k8s-0\" (UID: \"a668e185-3055-4ffa-95d4-055ca09a78b6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:04.872921 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:04.872832 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:05.034064 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.034041 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 20:44:05.036500 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:44:05.036474 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda668e185_3055_4ffa_95d4_055ca09a78b6.slice/crio-5324e0ab30830177db067c31df64171aa79487571cfff75ba8aa4375401c2a73 WatchSource:0}: Error finding container 5324e0ab30830177db067c31df64171aa79487571cfff75ba8aa4375401c2a73: Status 404 returned error can't find the container with id 5324e0ab30830177db067c31df64171aa79487571cfff75ba8aa4375401c2a73 Jul 10 20:44:05.163305 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.163273 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="231581db-c62c-4489-8087-16deddd47d73" path="/var/lib/kubelet/pods/231581db-c62c-4489-8087-16deddd47d73/volumes" Jul 10 20:44:05.507067 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.507034 2577 generic.go:356] "Generic (PLEG): container finished" podID="a668e185-3055-4ffa-95d4-055ca09a78b6" containerID="aee2abbc270112108bfdb526d1da2315f9ee1f474bc03e4780aec11e662d722b" exitCode=0 Jul 10 20:44:05.507442 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.507117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerDied","Data":"aee2abbc270112108bfdb526d1da2315f9ee1f474bc03e4780aec11e662d722b"} Jul 10 20:44:05.507442 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.507185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"5324e0ab30830177db067c31df64171aa79487571cfff75ba8aa4375401c2a73"} Jul 10 20:44:05.847227 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.847153 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:05.947066 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:05.947032 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:06.449602 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.449570 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:06.515226 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.515190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"89afc69caf62dacc40fcd51376e5cf0c0971ea7f878c2dfa3970bf41fcbd4854"} Jul 10 20:44:06.515583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.515258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"ca1bb2b4e5d28c154934ac0d4db9deeb598c110a3d1b43886b78edf095250568"} Jul 10 20:44:06.515583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.515283 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"9492ae03f3ee4d50f68036996ded063f8af6426895c18ed1b25ebdedcc3c8667"} Jul 10 20:44:06.515583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.515312 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"ee526e4e6a234a1c7702180332ea1f3f1754af855861492d244e5f00d9263cc5"} Jul 10 20:44:06.515583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.515337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"7a3ec6e682ed9b45c4849c5abd81deed16eef1d81c907a5bb5c36272eb4a998a"} Jul 10 20:44:06.515583 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.515358 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a668e185-3055-4ffa-95d4-055ca09a78b6","Type":"ContainerStarted","Data":"158ee8a1236a53a81f1b35b0a724e1d9dd622d8af886308e9110536be9117195"} Jul 10 20:44:06.543034 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.542970 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.523791074 podStartE2EDuration="2.542947843s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.156769ms" isStatefulPod=false podCreationTimestamp="2026-07-10 20:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:06.54051617 +0000 UTC m=+381.888825501" watchObservedRunningTime="2026-07-10 20:44:06.542947843 +0000 UTC m=+381.891257174" Jul 10 20:44:06.544650 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:06.544623 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:09.873304 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:09.873266 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:44:12.522119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:12.522084 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:12.623350 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:12.623317 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:12.678698 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:12.678666 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:12.894669 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:12.894572 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:12.994850 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:44:12.994814 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:00.086199 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:00.086167 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:03.918198 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:03.918167 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:04.014637 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:04.014603 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:04.873706 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:04.873648 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:45:04.889841 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:04.889819 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:45:05.757973 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:05.757944 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 20:45:06.205650 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:06.205565 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:22.354353 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:22.354271 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:22.409896 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:22.409862 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:30.488083 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:30.488045 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:31.734177 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:31.734087 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:33.943450 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:33.943419 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:34.043100 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:34.043070 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:35.134232 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:35.134203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:35.255617 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:35.255582 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:36.096326 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:36.096295 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:36.195085 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:36.195050 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:38.112216 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:38.112184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:38.494305 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:38.494275 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:40.933310 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:40.933276 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:40.988713 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:40.988681 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:44.022766 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:44.022733 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:44.124433 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:44.124383 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:44.510350 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:44.510309 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:44.566246 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:44.566204 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:46.704953 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:46.704920 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:46.949634 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:46.949598 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:47.051578 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:47.051554 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:53.423133 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:53.423095 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:53.527510 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:53.527482 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:55.175637 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:45:55.175607 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:07.973068 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:07.973036 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:08.066461 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:08.066430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:08.555176 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:08.555146 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:08.901562 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:08.901448 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.007024 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.006963 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.023726 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.023698 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28"] Jul 10 20:46:09.026980 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.026961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.029938 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.029916 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 20:46:09.030116 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.030099 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 20:46:09.030484 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.030467 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cndqb\"" Jul 10 20:46:09.030691 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.030650 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 20:46:09.030780 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.030700 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 20:46:09.046306 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.046279 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28"] Jul 10 20:46:09.138375 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.138351 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.152181 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.152117 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3061a25c-4747-4ff0-98b4-1a8776efc82f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.152181 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.152146 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3061a25c-4747-4ff0-98b4-1a8776efc82f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.152181 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.152171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhk9r\" (UniqueName: \"kubernetes.io/projected/3061a25c-4747-4ff0-98b4-1a8776efc82f-kube-api-access-xhk9r\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.152345 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.152190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3061a25c-4747-4ff0-98b4-1a8776efc82f-manifests\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.152345 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.152252 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3061a25c-4747-4ff0-98b4-1a8776efc82f-tmp\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.235287 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.235262 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.253422 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.253397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3061a25c-4747-4ff0-98b4-1a8776efc82f-tmp\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.253507 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.253457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3061a25c-4747-4ff0-98b4-1a8776efc82f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.253507 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.253481 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3061a25c-4747-4ff0-98b4-1a8776efc82f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.253589 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.253513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhk9r\" (UniqueName: \"kubernetes.io/projected/3061a25c-4747-4ff0-98b4-1a8776efc82f-kube-api-access-xhk9r\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.253589 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.253539 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3061a25c-4747-4ff0-98b4-1a8776efc82f-manifests\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.253921 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.253903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3061a25c-4747-4ff0-98b4-1a8776efc82f-manifests\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.255799 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.255780 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3061a25c-4747-4ff0-98b4-1a8776efc82f-tmp\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.256024 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.256006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3061a25c-4747-4ff0-98b4-1a8776efc82f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.256087 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.256047 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3061a25c-4747-4ff0-98b4-1a8776efc82f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.262365 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.262341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhk9r\" (UniqueName: \"kubernetes.io/projected/3061a25c-4747-4ff0-98b4-1a8776efc82f-kube-api-access-xhk9r\") pod \"opendatahub-operator-controller-manager-6558bffdbc-m6g28\" (UID: \"3061a25c-4747-4ff0-98b4-1a8776efc82f\") " pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.336566 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.336541 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:09.348806 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.348778 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.475993 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.475962 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.487395 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.487354 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28"] Jul 10 20:46:09.490790 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:46:09.490762 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3061a25c_4747_4ff0_98b4_1a8776efc82f.slice/crio-2caf1ed923255ab5da199dc65d8055855e8bf977f99dbc6b086f1528ae81d420 WatchSource:0}: Error finding container 2caf1ed923255ab5da199dc65d8055855e8bf977f99dbc6b086f1528ae81d420: Status 404 returned error can't find the container with id 2caf1ed923255ab5da199dc65d8055855e8bf977f99dbc6b086f1528ae81d420 Jul 10 20:46:09.568749 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.568721 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.599697 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.599647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7"] Jul 10 20:46:09.603881 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.603862 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.607182 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.607149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 20:46:09.607281 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.607159 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 20:46:09.607281 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.607162 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zr82z\"" Jul 10 20:46:09.607281 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.607228 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 20:46:09.607281 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.607273 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:46:09.607488 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.607367 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 20:46:09.615055 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.615038 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7"] Jul 10 20:46:09.721081 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.720999 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.758809 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.758779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkq5s\" (UniqueName: \"kubernetes.io/projected/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-kube-api-access-fkq5s\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.758929 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.758834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-cert\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.758929 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.758855 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-manager-config\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.758929 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.758872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-metrics-cert\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.847458 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.847433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.859605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.859583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-cert\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.859684 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.859614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-manager-config\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.859684 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.859631 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-metrics-cert\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.859759 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.859693 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkq5s\" (UniqueName: \"kubernetes.io/projected/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-kube-api-access-fkq5s\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.860331 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.860311 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-manager-config\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.862061 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.862039 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-cert\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.862167 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.862073 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-metrics-cert\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.869375 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.869353 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkq5s\" (UniqueName: \"kubernetes.io/projected/6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5-kube-api-access-fkq5s\") pod \"lws-controller-manager-66b48cb7bf-pk6s7\" (UID: \"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5\") " pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.913231 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.913208 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:09.993277 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:09.993242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" event={"ID":"3061a25c-4747-4ff0-98b4-1a8776efc82f","Type":"ContainerStarted","Data":"2caf1ed923255ab5da199dc65d8055855e8bf977f99dbc6b086f1528ae81d420"} Jul 10 20:46:10.055626 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:10.055595 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7"] Jul 10 20:46:10.057712 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:46:10.057681 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe05a50_dcd5_446c_bfa8_4a7ecd6c9fc5.slice/crio-40db8c4628ab3da670f1d2dd515f27c505626c4dc34eb22214b9d666bfb8d7cf WatchSource:0}: Error finding container 40db8c4628ab3da670f1d2dd515f27c505626c4dc34eb22214b9d666bfb8d7cf: Status 404 returned error can't find the container with id 40db8c4628ab3da670f1d2dd515f27c505626c4dc34eb22214b9d666bfb8d7cf Jul 10 20:46:11.003060 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:11.003022 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" event={"ID":"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5","Type":"ContainerStarted","Data":"40db8c4628ab3da670f1d2dd515f27c505626c4dc34eb22214b9d666bfb8d7cf"} Jul 10 20:46:14.017856 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:14.017816 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" event={"ID":"6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5","Type":"ContainerStarted","Data":"d8d107dd8a80595dc8a9d865ade787ebeb421fa8398563c2f55d90a436903c65"} Jul 10 20:46:14.018314 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:14.017911 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:14.019381 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:14.019349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" event={"ID":"3061a25c-4747-4ff0-98b4-1a8776efc82f","Type":"ContainerStarted","Data":"83359daf02fc0191263a96dd2d54a7edf99ba968588762ad924a60b1c7a9cf13"} Jul 10 20:46:14.033835 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:14.033779 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" podStartSLOduration=1.438934761 podStartE2EDuration="5.03376112s" totalImagesPullingTime="3.594826359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:14.033223879 +0000 UTC m=+509.381533208" watchObservedRunningTime="2026-07-10 20:46:14.03376112 +0000 UTC m=+509.382070453" Jul 10 20:46:15.025189 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:15.025153 2577 generic.go:356] "Generic (PLEG): container finished" podID="3061a25c-4747-4ff0-98b4-1a8776efc82f" containerID="83359daf02fc0191263a96dd2d54a7edf99ba968588762ad924a60b1c7a9cf13" exitCode=0 Jul 10 20:46:15.025549 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:15.025202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" event={"ID":"3061a25c-4747-4ff0-98b4-1a8776efc82f","Type":"ContainerDied","Data":"83359daf02fc0191263a96dd2d54a7edf99ba968588762ad924a60b1c7a9cf13"} Jul 10 20:46:16.031066 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:16.031036 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" event={"ID":"3061a25c-4747-4ff0-98b4-1a8776efc82f","Type":"ContainerStarted","Data":"e13690b8db42ab606ef0d847f4c5a8cc4eda8825bd06d36d0f331f66a50f50d1"} Jul 10 20:46:16.031466 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:16.031171 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:16.052813 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:16.052768 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" podStartSLOduration=2.554770128 podStartE2EDuration="8.052753728s" totalImagesPullingTime="4.479057096s" totalInitContainerRuntime="1.018926504s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:16.051031193 +0000 UTC m=+511.399340523" watchObservedRunningTime="2026-07-10 20:46:16.052753728 +0000 UTC m=+511.401063127" Jul 10 20:46:19.120877 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:19.120838 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:20.165693 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:20.165640 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:20.269641 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:20.269608 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:25.028025 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:25.027993 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66b48cb7bf-pk6s7" Jul 10 20:46:27.037237 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:27.037206 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6558bffdbc-m6g28" Jul 10 20:46:28.691811 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:28.691777 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:28.827002 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:28.826965 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:29.122993 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:29.122961 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:29.226449 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:29.226415 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:29.558102 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:29.558070 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:29.661971 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:29.661938 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:29.757580 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:29.757545 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:30.093632 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:30.093601 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:30.329377 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:30.329347 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:30.442987 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:30.442910 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:30.538538 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:30.538505 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:30.630005 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:30.629974 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:31.373957 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:31.373932 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:35.088537 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.088500 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:35.191520 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.191488 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:35.743971 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.743937 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694"] Jul 10 20:46:35.754982 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.754958 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.757824 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.757623 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 20:46:35.758342 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.758273 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2622h\"" Jul 10 20:46:35.759927 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.759904 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694"] Jul 10 20:46:35.777543 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777519 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777636 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777636 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777721 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777721 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777711 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777800 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777800 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32575539-5873-49c4-8da6-9e1aba76c3a6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777871 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfwmx\" (UniqueName: \"kubernetes.io/projected/32575539-5873-49c4-8da6-9e1aba76c3a6-kube-api-access-qfwmx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.777871 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.777829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.778549 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.778529 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:35.878810 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.878810 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878817 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878915 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.878981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32575539-5873-49c4-8da6-9e1aba76c3a6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfwmx\" (UniqueName: \"kubernetes.io/projected/32575539-5873-49c4-8da6-9e1aba76c3a6-kube-api-access-qfwmx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879394 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879230 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879582 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879524 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879582 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879854 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879768 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.879924 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.879900 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32575539-5873-49c4-8da6-9e1aba76c3a6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.881443 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.881420 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.881928 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.881894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.888428 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.888403 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32575539-5873-49c4-8da6-9e1aba76c3a6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.888720 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.888697 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfwmx\" (UniqueName: \"kubernetes.io/projected/32575539-5873-49c4-8da6-9e1aba76c3a6-kube-api-access-qfwmx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qn694\" (UID: \"32575539-5873-49c4-8da6-9e1aba76c3a6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:35.993149 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:35.993118 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:36.066177 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:36.066096 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:36.093605 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:36.093576 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:36.216046 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:36.216014 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694"] Jul 10 20:46:36.218940 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:46:36.218914 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32575539_5873_49c4_8da6_9e1aba76c3a6.slice/crio-8c53380d967d4bf7a46ceebee049a486c33189bb94956ce1d5a49b81fb456216 WatchSource:0}: Error finding container 8c53380d967d4bf7a46ceebee049a486c33189bb94956ce1d5a49b81fb456216: Status 404 returned error can't find the container with id 8c53380d967d4bf7a46ceebee049a486c33189bb94956ce1d5a49b81fb456216 Jul 10 20:46:37.121577 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:37.121479 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" event={"ID":"32575539-5873-49c4-8da6-9e1aba76c3a6","Type":"ContainerStarted","Data":"8c53380d967d4bf7a46ceebee049a486c33189bb94956ce1d5a49b81fb456216"} Jul 10 20:46:38.735239 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.735200 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:46:38.735512 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.735284 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:46:38.735512 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.735324 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:46:38.779673 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.779630 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq"] Jul 10 20:46:38.789883 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.789863 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.793267 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.792426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 20:46:38.793267 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.793034 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nfb7v\"" Jul 10 20:46:38.793267 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.793043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 20:46:38.793644 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.793538 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 20:46:38.798773 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.798716 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq"] Jul 10 20:46:38.808350 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daca5d24-2543-4659-bd4f-571d9e41e545-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.808463 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.808463 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/daca5d24-2543-4659-bd4f-571d9e41e545-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.808590 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.808590 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.808745 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808678 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/daca5d24-2543-4659-bd4f-571d9e41e545-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.808745 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.808715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjg2\" (UniqueName: \"kubernetes.io/projected/daca5d24-2543-4659-bd4f-571d9e41e545-kube-api-access-hjjg2\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.909557 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.909516 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/daca5d24-2543-4659-bd4f-571d9e41e545-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.909866 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.909565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjg2\" (UniqueName: \"kubernetes.io/projected/daca5d24-2543-4659-bd4f-571d9e41e545-kube-api-access-hjjg2\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.909866 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.909613 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daca5d24-2543-4659-bd4f-571d9e41e545-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.909866 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.909640 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.910058 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.909913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/daca5d24-2543-4659-bd4f-571d9e41e545-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.910058 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.909964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.910058 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.910010 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.910431 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.910403 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/daca5d24-2543-4659-bd4f-571d9e41e545-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.912838 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.912811 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.913073 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.913030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.913236 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.913209 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/daca5d24-2543-4659-bd4f-571d9e41e545-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.913916 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.913892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/daca5d24-2543-4659-bd4f-571d9e41e545-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.921908 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.921879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daca5d24-2543-4659-bd4f-571d9e41e545-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:38.925591 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:38.925561 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjg2\" (UniqueName: \"kubernetes.io/projected/daca5d24-2543-4659-bd4f-571d9e41e545-kube-api-access-hjjg2\") pod \"istiod-openshift-gateway-94bb8fbfd-nl4qq\" (UID: \"daca5d24-2543-4659-bd4f-571d9e41e545\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:39.103231 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:39.103195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:39.132559 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:39.132528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" event={"ID":"32575539-5873-49c4-8da6-9e1aba76c3a6","Type":"ContainerStarted","Data":"79407e309972d6f0123644751f06b7d5809910599e60c76623ae929026b7b6dd"} Jul 10 20:46:39.162013 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:39.160763 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" podStartSLOduration=1.646811158 podStartE2EDuration="4.160744985s" totalImagesPullingTime="2.513933827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:39.160149081 +0000 UTC m=+534.508458424" watchObservedRunningTime="2026-07-10 20:46:39.160744985 +0000 UTC m=+534.509054316" Jul 10 20:46:39.244084 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:39.244053 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq"] Jul 10 20:46:39.247247 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:46:39.247216 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaca5d24_2543_4659_bd4f_571d9e41e545.slice/crio-0155082825c86e3fdb28099877c69b6d0f1d3e7966111281270a7139b5cdbeee WatchSource:0}: Error finding container 0155082825c86e3fdb28099877c69b6d0f1d3e7966111281270a7139b5cdbeee: Status 404 returned error can't find the container with id 0155082825c86e3fdb28099877c69b6d0f1d3e7966111281270a7139b5cdbeee Jul 10 20:46:40.066318 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:40.066274 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:40.067530 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:40.067496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:40.071738 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:40.071711 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:40.139672 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:40.139615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" event={"ID":"daca5d24-2543-4659-bd4f-571d9e41e545","Type":"ContainerStarted","Data":"0155082825c86e3fdb28099877c69b6d0f1d3e7966111281270a7139b5cdbeee"} Jul 10 20:46:40.139964 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:40.139934 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:40.141202 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:40.141178 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qn694" Jul 10 20:46:41.164734 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:41.164621 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:41.283807 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:41.282570 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:41.487755 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:41.486994 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:46:41.487755 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:41.487091 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:46:41.614452 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:41.614417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:41.719061 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:41.719023 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:42.152835 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.152794 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" event={"ID":"daca5d24-2543-4659-bd4f-571d9e41e545","Type":"ContainerStarted","Data":"493c321cafefe17ddcee73db42a502a8dc4d4708a56af9a2612848bce61d96a8"} Jul 10 20:46:42.153036 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.152889 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:46:42.154561 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.154532 2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-nl4qq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 20:46:42.154683 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.154598 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" podUID="daca5d24-2543-4659-bd4f-571d9e41e545" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 20:46:42.174455 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.174385 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" podStartSLOduration=1.936940085 podStartE2EDuration="4.17436428s" totalImagesPullingTime="2.237424195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:42.172324047 +0000 UTC m=+537.520633378" watchObservedRunningTime="2026-07-10 20:46:42.17436428 +0000 UTC m=+537.522673611" Jul 10 20:46:42.497302 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.497229 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:42.589934 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:42.589901 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:43.167026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:46:43.167000 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nl4qq" Jul 10 20:47:01.391548 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:01.391512 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:09.155889 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:09.155850 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:26.201272 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:26.201235 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:26.252713 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:26.252681 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:30.138971 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:30.138931 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:30.229047 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:30.229017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:30.298519 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:30.298492 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:31.051305 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:31.051266 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:31.263083 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:47:31.263054 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:00.078355 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:00.078320 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:02.774447 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:02.774413 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:02.871502 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:02.871470 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:04.230164 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:04.230130 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:06.836828 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:06.836775 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:08.476479 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:08.476450 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:08.578511 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:08.578479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:08.678304 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:08.678276 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:09.204470 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:09.204441 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:09.297104 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:09.297070 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:09.874778 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:09.874745 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:11.415465 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:11.415429 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:11.665028 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:11.664998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:12.035035 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:12.035003 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:14.261705 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:14.261675 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:16.445714 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:16.445683 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:16.550981 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:16.550955 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:18.429981 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:18.429946 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:18.530306 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:18.530268 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:19.782789 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:19.782747 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:19.883051 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:19.883024 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:19.983180 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:19.983152 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:20.161504 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:20.161436 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:20.458480 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:20.458399 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:23.648070 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:23.648035 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:23.713416 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:23.713384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:31.850078 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:31.850047 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:31.947198 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:31.947163 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:34.444874 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:34.444850 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:34.543857 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:34.543823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:37.438721 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:37.438688 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:37.540921 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:37.540882 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:41.241926 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:41.241891 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:41.339222 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:41.339185 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:49.840764 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:49.840731 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:49.940168 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:49.940107 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:53.380331 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:53.380296 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:53.479261 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:48:53.479219 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:11.933372 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:11.933335 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh"] Jul 10 20:49:11.937051 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:11.937028 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:11.939390 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:11.939371 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8lwb7\"" Jul 10 20:49:11.948194 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:11.948168 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh"] Jul 10 20:49:11.960376 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:11.960353 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:12.046870 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.046837 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vd2f\" (UniqueName: \"kubernetes.io/projected/a1b1210f-ba9c-4541-9281-b9ae703ee904-kube-api-access-6vd2f\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.046874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.046899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047038 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.046919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047169 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.047040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047169 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.047082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047169 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.047115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.047169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.047276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.047197 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b1210f-ba9c-4541-9281-b9ae703ee904-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.134414 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.134380 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:12.148386 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b1210f-ba9c-4541-9281-b9ae703ee904-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148850 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vd2f\" (UniqueName: \"kubernetes.io/projected/a1b1210f-ba9c-4541-9281-b9ae703ee904-kube-api-access-6vd2f\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148850 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148576 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.148968 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.148935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.149049 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.149010 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.149049 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.149021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.149231 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.149207 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.149296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.149234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b1210f-ba9c-4541-9281-b9ae703ee904-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.150669 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.150634 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.151064 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.151042 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.157343 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.157322 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b1210f-ba9c-4541-9281-b9ae703ee904-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.157633 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.157614 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vd2f\" (UniqueName: \"kubernetes.io/projected/a1b1210f-ba9c-4541-9281-b9ae703ee904-kube-api-access-6vd2f\") pod \"maas-default-gateway-openshift-default-56654dcf7-6lprh\" (UID: \"a1b1210f-ba9c-4541-9281-b9ae703ee904\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.249483 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.249455 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:12.378805 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.378775 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh"] Jul 10 20:49:12.380890 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:49:12.380869 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b1210f_ba9c_4541_9281_b9ae703ee904.slice/crio-47f94c7c7d14e3c6b6728370eec2f1e8c27cbed6e90c4818fa6995a188c67948 WatchSource:0}: Error finding container 47f94c7c7d14e3c6b6728370eec2f1e8c27cbed6e90c4818fa6995a188c67948: Status 404 returned error can't find the container with id 47f94c7c7d14e3c6b6728370eec2f1e8c27cbed6e90c4818fa6995a188c67948 Jul 10 20:49:12.383405 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.383377 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:49:12.383526 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.383478 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:49:12.383587 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.383525 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:49:12.788972 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.788933 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" event={"ID":"a1b1210f-ba9c-4541-9281-b9ae703ee904","Type":"ContainerStarted","Data":"8bc176b928736be295b8e0703c7ae928c6e335b4a31b9355d0716ec074357f15"} Jul 10 20:49:12.789142 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.788993 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" event={"ID":"a1b1210f-ba9c-4541-9281-b9ae703ee904","Type":"ContainerStarted","Data":"47f94c7c7d14e3c6b6728370eec2f1e8c27cbed6e90c4818fa6995a188c67948"} Jul 10 20:49:12.810285 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:12.810237 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podStartSLOduration=1.81022163 podStartE2EDuration="1.81022163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:49:12.809136822 +0000 UTC m=+688.157446155" watchObservedRunningTime="2026-07-10 20:49:12.81022163 +0000 UTC m=+688.158530959" Jul 10 20:49:13.249992 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:13.249964 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:13.254871 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:13.254845 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:13.793519 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:13.793491 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:13.794341 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:13.794322 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:49:24.261901 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.261865 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:24.364950 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.364922 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:24.420142 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.420115 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:24.570430 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.570359 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:49:24.573826 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.573807 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.576258 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.576230 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dwf2s\"" Jul 10 20:49:24.576258 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.576251 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 20:49:24.577294 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.577278 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 20:49:24.577401 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.577309 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 20:49:24.583940 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.583922 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:49:24.609198 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.609176 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:49:24.664687 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.664640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8ee921b-aaca-4826-a64e-384f62d4724a-config-file\") pod \"limitador-limitador-54d6d988d5-c6gdk\" (UID: \"b8ee921b-aaca-4826-a64e-384f62d4724a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.664776 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.664711 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd4n4\" (UniqueName: \"kubernetes.io/projected/b8ee921b-aaca-4826-a64e-384f62d4724a-kube-api-access-jd4n4\") pod \"limitador-limitador-54d6d988d5-c6gdk\" (UID: \"b8ee921b-aaca-4826-a64e-384f62d4724a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.765964 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.765943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8ee921b-aaca-4826-a64e-384f62d4724a-config-file\") pod \"limitador-limitador-54d6d988d5-c6gdk\" (UID: \"b8ee921b-aaca-4826-a64e-384f62d4724a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.766071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.765992 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd4n4\" (UniqueName: \"kubernetes.io/projected/b8ee921b-aaca-4826-a64e-384f62d4724a-kube-api-access-jd4n4\") pod \"limitador-limitador-54d6d988d5-c6gdk\" (UID: \"b8ee921b-aaca-4826-a64e-384f62d4724a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.766497 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.766480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b8ee921b-aaca-4826-a64e-384f62d4724a-config-file\") pod \"limitador-limitador-54d6d988d5-c6gdk\" (UID: \"b8ee921b-aaca-4826-a64e-384f62d4724a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.773410 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.773383 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd4n4\" (UniqueName: \"kubernetes.io/projected/b8ee921b-aaca-4826-a64e-384f62d4724a-kube-api-access-jd4n4\") pod \"limitador-limitador-54d6d988d5-c6gdk\" (UID: \"b8ee921b-aaca-4826-a64e-384f62d4724a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:24.885574 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:24.885505 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:25.018746 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:25.018722 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:49:25.021198 ip-10-0-136-104 kubenswrapper[2577]: W0710 20:49:25.021171 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ee921b_aaca_4826_a64e_384f62d4724a.slice/crio-e63afba2b34e67bf3e427259db2d1957213e46e1734d00f676acfba11999dd82 WatchSource:0}: Error finding container e63afba2b34e67bf3e427259db2d1957213e46e1734d00f676acfba11999dd82: Status 404 returned error can't find the container with id e63afba2b34e67bf3e427259db2d1957213e46e1734d00f676acfba11999dd82 Jul 10 20:49:25.023367 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:25.023348 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:49:25.085533 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:25.085504 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:25.150371 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:25.150316 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:25.855218 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:25.855180 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" event={"ID":"b8ee921b-aaca-4826-a64e-384f62d4724a","Type":"ContainerStarted","Data":"e63afba2b34e67bf3e427259db2d1957213e46e1734d00f676acfba11999dd82"} Jul 10 20:49:27.873542 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:27.873459 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" event={"ID":"b8ee921b-aaca-4826-a64e-384f62d4724a","Type":"ContainerStarted","Data":"a2de182c07401b8ce148b231e8f56717ad548bd56f7d4bfe3259f06b99edb672"} Jul 10 20:49:27.873924 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:27.873596 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:27.888808 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:27.888752 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" podStartSLOduration=1.367317171 podStartE2EDuration="3.888737769s" totalImagesPullingTime="2.521420598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:49:27.887932032 +0000 UTC m=+703.236241368" watchObservedRunningTime="2026-07-10 20:49:27.888737769 +0000 UTC m=+703.237047099" Jul 10 20:49:38.878690 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:38.878583 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-c6gdk" Jul 10 20:49:40.292045 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:40.292013 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:40.385840 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:40.385808 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:40.807757 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:40.807724 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:47.274193 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:47.274162 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:47.376110 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:47.376072 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:53.195565 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:53.195518 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:53.593588 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:49:53.593552 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:00.013595 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:00.013563 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:00.114001 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:00.113967 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:00.168055 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:00.168024 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:00.429026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:00.428946 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:03.670561 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:03.670525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:03.773901 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:03.773870 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:36.706316 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:36.706277 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:36.808087 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:36.808055 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:37.111122 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:37.111081 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:37.322028 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:37.321999 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:37.696007 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:37.695971 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:37.795728 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:37.795648 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:38.066839 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:38.066801 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:50:44.834887 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:44.834838 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6lprh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 20:50:44.835298 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:44.834922 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podUID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 20:50:45.231821 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:45.231789 2577 generic.go:356] "Generic (PLEG): container finished" podID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerID="8bc176b928736be295b8e0703c7ae928c6e335b4a31b9355d0716ec074357f15" exitCode=0 Jul 10 20:50:45.232018 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:45.231858 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" event={"ID":"a1b1210f-ba9c-4541-9281-b9ae703ee904","Type":"ContainerDied","Data":"8bc176b928736be295b8e0703c7ae928c6e335b4a31b9355d0716ec074357f15"} Jul 10 20:50:45.232295 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:45.232281 2577 scope.go:117] "RemoveContainer" containerID="8bc176b928736be295b8e0703c7ae928c6e335b4a31b9355d0716ec074357f15" Jul 10 20:50:45.233320 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:45.233296 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:50:45.233383 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:45.233361 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:50:45.233419 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:45.233405 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 20:50:46.238740 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:46.238707 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" event={"ID":"a1b1210f-ba9c-4541-9281-b9ae703ee904","Type":"ContainerStarted","Data":"1c1a9fc2d445c9247b4714dbd9bb2150505e94332e81cf1eb923bd1af70ebdfc"} Jul 10 20:50:46.249796 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:46.249761 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:50:46.249969 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:46.249946 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6lprh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 10 20:50:46.250026 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:46.250008 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podUID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 10 20:50:47.087550 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:47.087521 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:47.185404 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:47.185349 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:47.250752 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:47.250710 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6lprh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 10 20:50:47.251244 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:47.250806 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podUID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 10 20:50:48.251846 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:48.251805 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6lprh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 10 20:50:48.252337 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:48.251883 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podUID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 10 20:50:49.250044 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:49.250005 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6lprh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 10 20:50:49.250328 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:49.250296 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podUID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 10 20:50:50.250886 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:50.250402 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6lprh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 10 20:50:50.250886 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:50.250466 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" podUID="a1b1210f-ba9c-4541-9281-b9ae703ee904" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 10 20:50:51.253696 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:51.253649 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:50:51.254092 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:51.253976 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:50:51.254600 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:51.254581 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6lprh" Jul 10 20:50:55.294617 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:55.294588 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:55.745119 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:50:55.745091 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:51:11.172005 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:11.171931 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:12.048364 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:12.048328 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:51:24.081253 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:24.081221 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:25.243031 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:25.243003 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:51:28.080114 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:28.080081 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:30.451563 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:30.451529 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:51:57.986198 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:51:57.986163 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:00.152105 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:00.152072 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:52:05.075397 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:05.075363 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:07.556294 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:07.556258 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:52:12.497323 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:12.497288 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:16.467825 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:16.467788 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:19.210719 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:19.210677 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:19.334705 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:52:19.334672 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:53:03.270713 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:53:03.270678 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:53:04.821627 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:53:04.821595 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:53:16.684801 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:53:16.684767 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:53:16.762857 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:53:16.762823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:54:11.084473 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:54:11.084396 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:54:11.181268 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:54:11.181237 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:54:12.233134 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:54:12.233098 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:54:12.333296 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:54:12.333261 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:54:40.447354 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:54:40.447318 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:54:49.847808 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:54:49.847774 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:55:01.165798 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:55:01.165768 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:55:08.850807 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:55:08.850764 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:55:20.260626 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:55:20.260585 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:55:22.523644 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:55:22.523592 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:55:24.144343 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:55:24.144309 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:55:50.658332 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:55:50.658294 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:56:02.950878 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:02.950847 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:56:11.000622 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:11.000580 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:11.096276 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:11.096242 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:11.229782 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:11.229751 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:13.853071 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:13.853042 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:16.870243 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:16.870199 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:16.971268 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:16.971240 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:17.071720 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:17.071684 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:17.218208 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:17.218131 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:22.969055 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:22.969017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:23.041379 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:23.041338 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:40.145269 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:40.145232 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:56:49.748054 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:56:49.748012 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:57:00.352121 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:57:00.352085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:57:06.044859 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:57:06.044830 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:57:16.156284 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:57:16.156209 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:57:25.647554 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:57:25.647517 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:57:38.344922 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:57:38.344882 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:57:48.042341 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:57:48.042306 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:58:09.455934 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:58:09.455897 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:58:21.544834 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:58:21.544799 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:58:44.152467 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:58:44.152428 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:58:54.760004 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:58:54.759965 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:59:09.756048 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:09.756014 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:59:09.949000 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:09.948957 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:59:11.883049 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:11.883015 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:59:11.981063 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:11.981029 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 20:59:21.158393 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:21.158363 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:59:30.707171 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:30.707135 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:59:40.458266 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:40.458231 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 20:59:49.767175 ip-10-0-136-104 kubenswrapper[2577]: I0710 20:59:49.767136 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:00:00.148744 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.148709 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl"] Jul 10 21:00:00.152420 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.152402 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" Jul 10 21:00:00.154865 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.154844 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:00:00.162593 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.162567 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" Jul 10 21:00:00.162724 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.162673 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl"] Jul 10 21:00:00.291629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.291606 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl"] Jul 10 21:00:00.294007 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:00:00.293978 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02547c5_1bd6_44f5_935f_f0abee814f83.slice/crio-749133c529b69be06635a1815e34304d9338911c928efe34a62bb37a009c4db6 WatchSource:0}: Error finding container 749133c529b69be06635a1815e34304d9338911c928efe34a62bb37a009c4db6: Status 404 returned error can't find the container with id 749133c529b69be06635a1815e34304d9338911c928efe34a62bb37a009c4db6 Jul 10 21:00:00.296123 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.296108 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 21:00:00.702854 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:00.702819 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" event={"ID":"b02547c5-1bd6-44f5-935f-f0abee814f83","Type":"ContainerStarted","Data":"749133c529b69be06635a1815e34304d9338911c928efe34a62bb37a009c4db6"} Jul 10 21:00:20.795732 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:20.795688 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" event={"ID":"b02547c5-1bd6-44f5-935f-f0abee814f83","Type":"ContainerStarted","Data":"a523311b1e2c628e90a4e72b79703d733c9164c4c753f88da9f7b9f11bc5932b"} Jul 10 21:00:20.811105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:20.811052 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" podStartSLOduration=0.834210479 podStartE2EDuration="20.81103044s" totalImagesPullingTime="19.976819961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:00:20.809764596 +0000 UTC m=+1356.158073924" watchObservedRunningTime="2026-07-10 21:00:20.81103044 +0000 UTC m=+1356.159339772" Jul 10 21:00:21.801757 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:21.801701 2577 generic.go:356] "Generic (PLEG): container finished" podID="b02547c5-1bd6-44f5-935f-f0abee814f83" containerID="a523311b1e2c628e90a4e72b79703d733c9164c4c753f88da9f7b9f11bc5932b" exitCode=0 Jul 10 21:00:21.802128 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:21.801767 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" event={"ID":"b02547c5-1bd6-44f5-935f-f0abee814f83","Type":"ContainerDied","Data":"a523311b1e2c628e90a4e72b79703d733c9164c4c753f88da9f7b9f11bc5932b"} Jul 10 21:00:22.939034 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:22.939003 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" Jul 10 21:00:23.811592 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:23.811560 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" Jul 10 21:00:23.811592 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:23.811583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728620-t7ntl" event={"ID":"b02547c5-1bd6-44f5-935f-f0abee814f83","Type":"ContainerDied","Data":"749133c529b69be06635a1815e34304d9338911c928efe34a62bb37a009c4db6"} Jul 10 21:00:23.811794 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:23.811632 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="749133c529b69be06635a1815e34304d9338911c928efe34a62bb37a009c4db6" Jul 10 21:00:40.697884 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:00:40.697815 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:01:25.141457 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:01:25.141422 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:01:25.242321 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:01:25.242280 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:01:37.050616 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:01:37.050539 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:01:45.948440 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:01:45.948403 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:01:56.859988 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:01:56.859946 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:02:07.962369 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:02:07.962331 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:02:18.945280 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:02:18.945247 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:02:29.349990 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:02:29.349957 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:02:39.659117 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:02:39.659080 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:02:48.858310 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:02:48.858270 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:02:59.850819 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:02:59.850773 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:03:10.752752 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:03:10.752674 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:03:22.844470 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:03:22.844437 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:03:44.453647 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:03:44.453610 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:03:53.847114 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:03:53.847080 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:04:03.753260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:04:03.753224 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:04:16.547597 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:04:16.547563 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:04:32.654434 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:04:32.654390 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:04:41.950703 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:04:41.950613 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:04:51.147893 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:04:51.147837 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:04:59.247774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:04:59.247732 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:05:08.553802 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:05:08.553761 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:05:18.045647 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:05:18.045607 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:05:31.442278 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:05:31.442240 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:05:40.556454 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:05:40.556418 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:05:50.053963 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:05:50.053924 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:06:02.349204 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:06:02.349154 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:06:11.265144 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:06:11.260115 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:06:18.050474 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:06:18.050439 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:06:35.665326 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:06:35.665287 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:06:47.649323 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:06:47.649287 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:06:58.847219 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:06:58.847180 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:07:01.576411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:01.576381 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:08.001400 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:08.001367 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:12.048686 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:12.048632 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:07:14.785979 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:14.785948 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:16.281567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:16.281530 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:21.462846 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:21.462809 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:21.649926 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:21.649896 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:07:24.521131 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:24.521096 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:24.620717 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:24.620688 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:25.000128 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:25.000102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:30.734545 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.734509 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:30.804382 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.804348 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:30.914181 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.914142 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww"] Jul 10 21:07:30.914511 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.914496 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b02547c5-1bd6-44f5-935f-f0abee814f83" containerName="cleanup" Jul 10 21:07:30.914511 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.914511 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b02547c5-1bd6-44f5-935f-f0abee814f83" containerName="cleanup" Jul 10 21:07:30.914692 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.914601 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b02547c5-1bd6-44f5-935f-f0abee814f83" containerName="cleanup" Jul 10 21:07:30.917709 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.917689 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.920603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.920584 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-35794783-openshift-default-dockercfg-r8nf5\"" Jul 10 21:07:30.937790 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.937767 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww"] Jul 10 21:07:30.945246 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.945225 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:30.954285 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-envoy\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954389 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-data\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954389 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-token\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954389 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a0c59b1-86d9-477f-b731-dc05183949ae-istiod-ca-cert\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954500 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954443 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-socket\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954500 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-credential-socket\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954500 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-certs\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954609 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954513 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-podinfo\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:30.954609 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:30.954533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjdxf\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-kube-api-access-fjdxf\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.008323 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.008293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:31.055286 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-envoy\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055286 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-data\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-token\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a0c59b1-86d9-477f-b731-dc05183949ae-istiod-ca-cert\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-socket\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-credential-socket\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-certs\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055469 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-podinfo\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.055535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.055493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjdxf\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-kube-api-access-fjdxf\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.056067 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.056015 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-certs\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.056067 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.056041 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-data\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.056260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.056068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-credential-socket\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.056260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.056112 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-socket\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.056260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.056140 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a0c59b1-86d9-477f-b731-dc05183949ae-istiod-ca-cert\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.057720 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.057701 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-envoy\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.058108 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.058092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-podinfo\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.064155 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.064135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-token\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.064411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.064389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjdxf\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-kube-api-access-fjdxf\") pod \"e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.229248 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.229216 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:31.351764 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.351734 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:07:31.365722 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.365538 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww"] Jul 10 21:07:31.368833 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:07:31.368164 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0c59b1_86d9_477f_b731_dc05183949ae.slice/crio-9473b74e4237dac9b9ee3e198f3c7f7cbaecd2f52e31c89630d827f3ecbeca40 WatchSource:0}: Error finding container 9473b74e4237dac9b9ee3e198f3c7f7cbaecd2f52e31c89630d827f3ecbeca40: Status 404 returned error can't find the container with id 9473b74e4237dac9b9ee3e198f3c7f7cbaecd2f52e31c89630d827f3ecbeca40 Jul 10 21:07:31.370871 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.370828 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:07:31.370976 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.370913 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:07:31.370976 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.370954 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:07:31.768248 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.768210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" event={"ID":"8a0c59b1-86d9-477f-b731-dc05183949ae","Type":"ContainerStarted","Data":"80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254"} Jul 10 21:07:31.768760 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.768273 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" event={"ID":"8a0c59b1-86d9-477f-b731-dc05183949ae","Type":"ContainerStarted","Data":"9473b74e4237dac9b9ee3e198f3c7f7cbaecd2f52e31c89630d827f3ecbeca40"} Jul 10 21:07:31.792304 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:31.792249 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" podStartSLOduration=1.792233293 podStartE2EDuration="1.792233293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:07:31.79026357 +0000 UTC m=+1787.138572903" watchObservedRunningTime="2026-07-10 21:07:31.792233293 +0000 UTC m=+1787.140542622" Jul 10 21:07:32.058720 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.058569 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:32.145394 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.145348 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:32.217645 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.217592 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:32.229466 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.229440 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:32.234418 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.234394 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:32.317833 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.317702 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:32.774038 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.774006 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:32.775029 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.775011 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:32.822673 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.822636 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:32.922707 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:32.922674 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:34.929670 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:34.929627 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:37.174765 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.174691 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4"] Jul 10 21:07:37.178393 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.178375 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:37.181087 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.181072 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:07:37.181159 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.181102 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:07:37.186305 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.186280 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4"] Jul 10 21:07:37.214641 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.214609 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dea44-a62e-4f73-8d39-3926b570d162-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35794783-ntmj4\" (UID: \"b37dea44-a62e-4f73-8d39-3926b570d162\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:37.315141 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.315100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dea44-a62e-4f73-8d39-3926b570d162-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35794783-ntmj4\" (UID: \"b37dea44-a62e-4f73-8d39-3926b570d162\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:37.315741 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.315720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dea44-a62e-4f73-8d39-3926b570d162-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35794783-ntmj4\" (UID: \"b37dea44-a62e-4f73-8d39-3926b570d162\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:37.489816 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.489733 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:37.669153 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.669130 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4"] Jul 10 21:07:37.797803 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:37.797768 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" event={"ID":"b37dea44-a62e-4f73-8d39-3926b570d162","Type":"ContainerStarted","Data":"fc748eec5277c891afa4b5b6f1a39a2890efc78b9b0588cbd27f89d661a4a4de"} Jul 10 21:07:38.803991 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:38.803898 2577 generic.go:356] "Generic (PLEG): container finished" podID="b37dea44-a62e-4f73-8d39-3926b570d162" containerID="22c16346e09d7888fd0a40ab859c3a8184cc79ae8893d485cadc8e94d8a14d91" exitCode=7 Jul 10 21:07:38.804357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:38.803987 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" event={"ID":"b37dea44-a62e-4f73-8d39-3926b570d162","Type":"ContainerDied","Data":"22c16346e09d7888fd0a40ab859c3a8184cc79ae8893d485cadc8e94d8a14d91"} Jul 10 21:07:38.804357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:38.804151 2577 scope.go:117] "RemoveContainer" containerID="22c16346e09d7888fd0a40ab859c3a8184cc79ae8893d485cadc8e94d8a14d91" Jul 10 21:07:39.810646 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:39.810614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" event={"ID":"b37dea44-a62e-4f73-8d39-3926b570d162","Type":"ContainerStarted","Data":"f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4"} Jul 10 21:07:39.831996 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:39.831920 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" podStartSLOduration=2.831885431 podStartE2EDuration="2.831885431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:07:39.828735261 +0000 UTC m=+1795.177044594" watchObservedRunningTime="2026-07-10 21:07:39.831885431 +0000 UTC m=+1795.180194762" Jul 10 21:07:39.988720 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:39.988633 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:07:40.817641 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:40.817604 2577 generic.go:356] "Generic (PLEG): container finished" podID="b37dea44-a62e-4f73-8d39-3926b570d162" containerID="f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4" exitCode=7 Jul 10 21:07:40.818131 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:40.817689 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" event={"ID":"b37dea44-a62e-4f73-8d39-3926b570d162","Type":"ContainerDied","Data":"f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4"} Jul 10 21:07:40.818131 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:40.817749 2577 scope.go:117] "RemoveContainer" containerID="22c16346e09d7888fd0a40ab859c3a8184cc79ae8893d485cadc8e94d8a14d91" Jul 10 21:07:40.818131 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:40.818016 2577 scope.go:117] "RemoveContainer" containerID="f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4" Jul 10 21:07:40.818258 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:07:40.818222 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-35794783-ntmj4_odh-ai-gateway-infra(b37dea44-a62e-4f73-8d39-3926b570d162)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" Jul 10 21:07:41.824359 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:41.824330 2577 scope.go:117] "RemoveContainer" containerID="f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4" Jul 10 21:07:41.824746 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:07:41.824532 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-35794783-ntmj4_odh-ai-gateway-infra(b37dea44-a62e-4f73-8d39-3926b570d162)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" Jul 10 21:07:44.839068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:44.839028 2577 generic.go:356] "Generic (PLEG): container finished" podID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerID="80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254" exitCode=0 Jul 10 21:07:44.839068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:44.839070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" event={"ID":"8a0c59b1-86d9-477f-b731-dc05183949ae","Type":"ContainerDied","Data":"80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254"} Jul 10 21:07:44.839495 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:44.839453 2577 scope.go:117] "RemoveContainer" containerID="80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254" Jul 10 21:07:44.840498 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:44.840468 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:07:44.840570 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:44.840554 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:07:44.840606 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:44.840598 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:07:45.849570 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:45.849515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" event={"ID":"8a0c59b1-86d9-477f-b731-dc05183949ae","Type":"ContainerStarted","Data":"5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda"} Jul 10 21:07:46.230376 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:46.230282 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:46.231991 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:46.231958 2577 patch_prober.go:28] interesting pod/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 10 21:07:46.232211 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:46.232185 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 10 21:07:47.229929 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:47.229890 2577 patch_prober.go:28] interesting pod/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 10 21:07:47.230537 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:47.230501 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 10 21:07:47.774761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:47.774727 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:48.230693 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:48.230576 2577 patch_prober.go:28] interesting pod/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 10 21:07:48.230693 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:48.230641 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 10 21:07:49.230379 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:49.230331 2577 patch_prober.go:28] interesting pod/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 10 21:07:49.230724 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:49.230689 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 10 21:07:50.234099 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:50.234067 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:50.235172 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:50.235151 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:07:54.158721 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:54.158689 2577 scope.go:117] "RemoveContainer" containerID="f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4" Jul 10 21:07:54.900263 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:54.900230 2577 generic.go:356] "Generic (PLEG): container finished" podID="b37dea44-a62e-4f73-8d39-3926b570d162" containerID="f00dfa2bd0a8034e64d398efe5ca328a735a211833b043aa84a12c207ff0b936" exitCode=0 Jul 10 21:07:54.900411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:54.900308 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" event={"ID":"b37dea44-a62e-4f73-8d39-3926b570d162","Type":"ContainerDied","Data":"f00dfa2bd0a8034e64d398efe5ca328a735a211833b043aa84a12c207ff0b936"} Jul 10 21:07:54.900411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:54.900393 2577 scope.go:117] "RemoveContainer" containerID="f567d86bc290900eab926480994983a3f41d966d7cae1819b34324fdc7d1bda4" Jul 10 21:07:55.926527 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:55.926500 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4"] Jul 10 21:07:56.049851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.049829 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:56.205342 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.205261 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dea44-a62e-4f73-8d39-3926b570d162-maas-api-service-ca\") pod \"b37dea44-a62e-4f73-8d39-3926b570d162\" (UID: \"b37dea44-a62e-4f73-8d39-3926b570d162\") " Jul 10 21:07:56.205579 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.205555 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b37dea44-a62e-4f73-8d39-3926b570d162-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b37dea44-a62e-4f73-8d39-3926b570d162" (UID: "b37dea44-a62e-4f73-8d39-3926b570d162"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:07:56.306650 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.306614 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dea44-a62e-4f73-8d39-3926b570d162-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:07:56.913121 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.913086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" event={"ID":"b37dea44-a62e-4f73-8d39-3926b570d162","Type":"ContainerDied","Data":"fc748eec5277c891afa4b5b6f1a39a2890efc78b9b0588cbd27f89d661a4a4de"} Jul 10 21:07:56.913316 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.913142 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc748eec5277c891afa4b5b6f1a39a2890efc78b9b0588cbd27f89d661a4a4de" Jul 10 21:07:56.913316 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.913103 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4" Jul 10 21:07:56.945488 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.945463 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4"] Jul 10 21:07:56.947603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:56.947578 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-ntmj4"] Jul 10 21:07:57.164458 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:07:57.164383 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" path="/var/lib/kubelet/pods/b37dea44-a62e-4f73-8d39-3926b570d162/volumes" Jul 10 21:08:14.046830 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:14.046791 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:14.101059 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:14.101032 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:27.221416 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221385 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph"] Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221742 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221754 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221773 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221779 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221832 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221839 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.221882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.221847 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:27.224826 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.224809 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:27.227330 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.227314 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:08:27.227417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.227399 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:08:27.231488 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.231465 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph"] Jul 10 21:08:27.254930 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.254531 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531a691-9c2d-4b88-8567-2f4207708147-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35794783-jppph\" (UID: \"7531a691-9c2d-4b88-8567-2f4207708147\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:27.356271 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.356240 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531a691-9c2d-4b88-8567-2f4207708147-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35794783-jppph\" (UID: \"7531a691-9c2d-4b88-8567-2f4207708147\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:27.356888 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.356868 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531a691-9c2d-4b88-8567-2f4207708147-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-35794783-jppph\" (UID: \"7531a691-9c2d-4b88-8567-2f4207708147\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:27.537351 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.537309 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:27.665982 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:27.665956 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph"] Jul 10 21:08:27.668040 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:08:27.668011 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7531a691_9c2d_4b88_8567_2f4207708147.slice/crio-a626a253f8112e3c35778da5c3442c5962c305391f63b29e10c83979711139b1 WatchSource:0}: Error finding container a626a253f8112e3c35778da5c3442c5962c305391f63b29e10c83979711139b1: Status 404 returned error can't find the container with id a626a253f8112e3c35778da5c3442c5962c305391f63b29e10c83979711139b1 Jul 10 21:08:28.061979 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:28.061947 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" event={"ID":"7531a691-9c2d-4b88-8567-2f4207708147","Type":"ContainerStarted","Data":"3df833346ac703f7fe3a9c6f27848c7a016464a16f42032b3da2eeff50641124"} Jul 10 21:08:28.062105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:28.061997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" event={"ID":"7531a691-9c2d-4b88-8567-2f4207708147","Type":"ContainerStarted","Data":"a626a253f8112e3c35778da5c3442c5962c305391f63b29e10c83979711139b1"} Jul 10 21:08:28.078777 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:28.078732 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" podStartSLOduration=1.078716184 podStartE2EDuration="1.078716184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:08:28.076946934 +0000 UTC m=+1843.425256264" watchObservedRunningTime="2026-07-10 21:08:28.078716184 +0000 UTC m=+1843.427025513" Jul 10 21:08:29.068165 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:29.068129 2577 generic.go:356] "Generic (PLEG): container finished" podID="7531a691-9c2d-4b88-8567-2f4207708147" containerID="3df833346ac703f7fe3a9c6f27848c7a016464a16f42032b3da2eeff50641124" exitCode=0 Jul 10 21:08:29.068722 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:29.068177 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" event={"ID":"7531a691-9c2d-4b88-8567-2f4207708147","Type":"ContainerDied","Data":"3df833346ac703f7fe3a9c6f27848c7a016464a16f42032b3da2eeff50641124"} Jul 10 21:08:30.201598 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:30.201569 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:30.282258 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:30.282228 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531a691-9c2d-4b88-8567-2f4207708147-maas-api-service-ca\") pod \"7531a691-9c2d-4b88-8567-2f4207708147\" (UID: \"7531a691-9c2d-4b88-8567-2f4207708147\") " Jul 10 21:08:30.282568 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:30.282545 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7531a691-9c2d-4b88-8567-2f4207708147-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7531a691-9c2d-4b88-8567-2f4207708147" (UID: "7531a691-9c2d-4b88-8567-2f4207708147"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:08:30.383857 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:30.383777 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7531a691-9c2d-4b88-8567-2f4207708147-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:31.079544 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:31.079510 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" Jul 10 21:08:31.079544 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:31.079535 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-35794783-jppph" event={"ID":"7531a691-9c2d-4b88-8567-2f4207708147","Type":"ContainerDied","Data":"a626a253f8112e3c35778da5c3442c5962c305391f63b29e10c83979711139b1"} Jul 10 21:08:31.079775 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:31.079585 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a626a253f8112e3c35778da5c3442c5962c305391f63b29e10c83979711139b1" Jul 10 21:08:37.741343 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:37.741306 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:40.463340 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:40.463302 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:08:42.314336 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:42.314302 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:48.552482 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:48.552449 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:48.644781 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:48.644746 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:52.514600 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:52.514563 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww"] Jul 10 21:08:52.515070 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:52.514782 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" containerID="cri-o://5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda" gracePeriod=30 Jul 10 21:08:52.543054 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:52.543025 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:52.919830 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:52.919743 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:53.015685 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.015631 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:53.176442 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.176348 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp"] Jul 10 21:08:53.176874 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.176851 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:53.176874 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.176874 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37dea44-a62e-4f73-8d39-3926b570d162" containerName="revoke-keys" Jul 10 21:08:53.177050 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.176886 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7531a691-9c2d-4b88-8567-2f4207708147" containerName="revoke-keys" Jul 10 21:08:53.177050 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.176895 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7531a691-9c2d-4b88-8567-2f4207708147" containerName="revoke-keys" Jul 10 21:08:53.177050 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.177020 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7531a691-9c2d-4b88-8567-2f4207708147" containerName="revoke-keys" Jul 10 21:08:53.180156 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.180135 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.183858 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.183834 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6fde404a-gw-openshift-default-dockercfg-cvbrm\"" Jul 10 21:08:53.212346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.212309 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp"] Jul 10 21:08:53.230937 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.230904 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:53.283423 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11cd7a03-1814-4249-87f8-df8feac03c57-istio-podinfo\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283608 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2sqt\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-kube-api-access-q2sqt\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283608 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-certs\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283608 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-envoy\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283820 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11cd7a03-1814-4249-87f8-df8feac03c57-istiod-ca-cert\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283820 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283796 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-socket\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283925 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-data\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.283925 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-credential-socket\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.284028 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.283920 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-istio-token\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sqt\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-kube-api-access-q2sqt\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-certs\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-envoy\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11cd7a03-1814-4249-87f8-df8feac03c57-istiod-ca-cert\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-socket\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385569 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-data\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-credential-socket\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-istio-token\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385681 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11cd7a03-1814-4249-87f8-df8feac03c57-istio-podinfo\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.385979 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.385859 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-certs\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.386207 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.386181 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11cd7a03-1814-4249-87f8-df8feac03c57-istiod-ca-cert\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.386350 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.386260 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-credential-socket\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.386350 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.386330 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-socket\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.386453 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.386385 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-data\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.387844 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.387817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-envoy\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.388119 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.388101 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11cd7a03-1814-4249-87f8-df8feac03c57-istio-podinfo\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.399201 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.399180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2sqt\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-kube-api-access-q2sqt\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.401850 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.401828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-istio-token\") pod \"e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.490904 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.490880 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:53.628929 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.628901 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp"] Jul 10 21:08:53.630681 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:08:53.630639 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cd7a03_1814_4249_87f8_df8feac03c57.slice/crio-b601d5375c88de9fc7f9dca85447dad9ab4a24951ec80c86a29280685a09aa11 WatchSource:0}: Error finding container b601d5375c88de9fc7f9dca85447dad9ab4a24951ec80c86a29280685a09aa11: Status 404 returned error can't find the container with id b601d5375c88de9fc7f9dca85447dad9ab4a24951ec80c86a29280685a09aa11 Jul 10 21:08:53.632898 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.632867 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:08:53.632992 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.632948 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:08:53.633056 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:53.632990 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:08:54.186710 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.186675 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" event={"ID":"11cd7a03-1814-4249-87f8-df8feac03c57","Type":"ContainerStarted","Data":"7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62"} Jul 10 21:08:54.186899 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.186731 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" event={"ID":"11cd7a03-1814-4249-87f8-df8feac03c57","Type":"ContainerStarted","Data":"b601d5375c88de9fc7f9dca85447dad9ab4a24951ec80c86a29280685a09aa11"} Jul 10 21:08:54.207869 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.207809 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" podStartSLOduration=1.2077869589999999 podStartE2EDuration="1.207786959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:08:54.206600038 +0000 UTC m=+1869.554909406" watchObservedRunningTime="2026-07-10 21:08:54.207786959 +0000 UTC m=+1869.556096290" Jul 10 21:08:54.491297 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.491262 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:54.495916 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.495888 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:54.496493 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.496474 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:54.795891 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.795792 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:54.898870 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:54.898841 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:55.191597 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:55.191509 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:55.192578 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:55.192556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:08:55.293697 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:55.293630 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:55.395525 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:55.395486 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:56.554681 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:56.554629 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:08:57.793617 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.793591 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.936954 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjdxf\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-kube-api-access-fjdxf\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937020 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-credential-socket\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937050 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-socket\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937076 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-podinfo\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937144 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a0c59b1-86d9-477f-b731-dc05183949ae-istiod-ca-cert\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937200 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-envoy\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937224 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-token\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937263 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-certs\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.937684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.937344 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-data\") pod \"8a0c59b1-86d9-477f-b731-dc05183949ae\" (UID: \"8a0c59b1-86d9-477f-b731-dc05183949ae\") " Jul 10 21:08:57.941688 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.938648 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:57.945678 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.943031 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a0c59b1-86d9-477f-b731-dc05183949ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:08:57.945678 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.943294 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:57.945678 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.943460 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:57.949678 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.946896 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:57.956003 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.955312 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-kube-api-access-fjdxf" (OuterVolumeSpecName: "kube-api-access-fjdxf") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "kube-api-access-fjdxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:08:57.956003 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.955406 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:08:57.956003 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.955755 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:57.956282 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:57.956032 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a0c59b1-86d9-477f-b731-dc05183949ae" (UID: "8a0c59b1-86d9-477f-b731-dc05183949ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038799 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a0c59b1-86d9-477f-b731-dc05183949ae-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038853 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038865 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038876 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038887 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038898 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjdxf\" (UniqueName: \"kubernetes.io/projected/8a0c59b1-86d9-477f-b731-dc05183949ae-kube-api-access-fjdxf\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.038913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038911 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.039357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038923 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a0c59b1-86d9-477f-b731-dc05183949ae-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.039357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.038935 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a0c59b1-86d9-477f-b731-dc05183949ae-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:08:58.207068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.207028 2577 generic.go:356] "Generic (PLEG): container finished" podID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerID="5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda" exitCode=0 Jul 10 21:08:58.207255 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.207084 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" Jul 10 21:08:58.207255 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.207106 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" event={"ID":"8a0c59b1-86d9-477f-b731-dc05183949ae","Type":"ContainerDied","Data":"5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda"} Jul 10 21:08:58.207255 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.207162 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww" event={"ID":"8a0c59b1-86d9-477f-b731-dc05183949ae","Type":"ContainerDied","Data":"9473b74e4237dac9b9ee3e198f3c7f7cbaecd2f52e31c89630d827f3ecbeca40"} Jul 10 21:08:58.207255 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.207187 2577 scope.go:117] "RemoveContainer" containerID="5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda" Jul 10 21:08:58.216079 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.216058 2577 scope.go:117] "RemoveContainer" containerID="80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254" Jul 10 21:08:58.224226 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.224209 2577 scope.go:117] "RemoveContainer" containerID="5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda" Jul 10 21:08:58.224493 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:08:58.224473 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda\": container with ID starting with 5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda not found: ID does not exist" containerID="5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda" Jul 10 21:08:58.224539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.224503 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda"} err="failed to get container status \"5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda\": rpc error: code = NotFound desc = could not find container \"5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda\": container with ID starting with 5eae402da0f689f79d21d2b1fca21699f4a31cee66c59fe0808f1aeb8d688eda not found: ID does not exist" Jul 10 21:08:58.224539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.224520 2577 scope.go:117] "RemoveContainer" containerID="80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254" Jul 10 21:08:58.224894 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:08:58.224868 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254\": container with ID starting with 80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254 not found: ID does not exist" containerID="80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254" Jul 10 21:08:58.224981 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.224902 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254"} err="failed to get container status \"80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254\": rpc error: code = NotFound desc = could not find container \"80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254\": container with ID starting with 80fd7977d09c18161304942f1ba87fea0418ea83af3f47149b131ddb1fb02254 not found: ID does not exist" Jul 10 21:08:58.236291 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.236254 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww"] Jul 10 21:08:58.244676 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.244599 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-35794783-openshift-default-dfc7b89b7-xfkww"] Jul 10 21:08:58.922577 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.922541 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz"] Jul 10 21:08:58.923069 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.922975 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" Jul 10 21:08:58.923069 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.922988 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" Jul 10 21:08:58.923069 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.923008 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" Jul 10 21:08:58.923069 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.923015 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" Jul 10 21:08:58.923069 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.923071 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" Jul 10 21:08:58.923231 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.923081 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" containerName="istio-proxy" Jul 10 21:08:58.927477 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.927458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:08:58.930552 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.930103 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:08:58.930552 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.930367 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:08:58.935513 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.935481 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz"] Jul 10 21:08:58.961758 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:58.961715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz\" (UID: \"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:08:59.062758 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.062719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz\" (UID: \"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:08:59.063557 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.063527 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz\" (UID: \"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:08:59.080382 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.080356 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:59.147680 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.145718 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:59.167490 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.167453 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a0c59b1-86d9-477f-b731-dc05183949ae" path="/var/lib/kubelet/pods/8a0c59b1-86d9-477f-b731-dc05183949ae/volumes" Jul 10 21:08:59.262853 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.262795 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:08:59.418512 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:08:59.413267 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz"] Jul 10 21:09:00.218993 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:00.218954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" event={"ID":"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d","Type":"ContainerStarted","Data":"a892d8a93f766c7df905e2d80f77b0c9e1fc65e0a885fa9e1634e9b722141ad8"} Jul 10 21:09:00.219373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:00.219008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" event={"ID":"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d","Type":"ContainerStarted","Data":"ff4e6d2f5d1e7cb0be1a70ed3043cc1949912ed2caef004f99a63a5977991c83"} Jul 10 21:09:01.224870 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:01.224839 2577 generic.go:356] "Generic (PLEG): container finished" podID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerID="a892d8a93f766c7df905e2d80f77b0c9e1fc65e0a885fa9e1634e9b722141ad8" exitCode=7 Jul 10 21:09:01.225230 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:01.224877 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" event={"ID":"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d","Type":"ContainerDied","Data":"a892d8a93f766c7df905e2d80f77b0c9e1fc65e0a885fa9e1634e9b722141ad8"} Jul 10 21:09:01.225230 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:01.225072 2577 scope.go:117] "RemoveContainer" containerID="a892d8a93f766c7df905e2d80f77b0c9e1fc65e0a885fa9e1634e9b722141ad8" Jul 10 21:09:02.232040 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:02.231958 2577 generic.go:356] "Generic (PLEG): container finished" podID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerID="0d8f3afa46d0c361e0d2c181efbd9c61b287e0d67d6c7c7f518b52a6e3a64a34" exitCode=7 Jul 10 21:09:02.232464 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:02.232078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" event={"ID":"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d","Type":"ContainerDied","Data":"0d8f3afa46d0c361e0d2c181efbd9c61b287e0d67d6c7c7f518b52a6e3a64a34"} Jul 10 21:09:02.232464 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:02.232142 2577 scope.go:117] "RemoveContainer" containerID="a892d8a93f766c7df905e2d80f77b0c9e1fc65e0a885fa9e1634e9b722141ad8" Jul 10 21:09:02.232464 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:02.232342 2577 scope.go:117] "RemoveContainer" containerID="0d8f3afa46d0c361e0d2c181efbd9c61b287e0d67d6c7c7f518b52a6e3a64a34" Jul 10 21:09:02.232623 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:09:02.232604 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz_odh-ai-gateway-infra(9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" Jul 10 21:09:02.560147 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:02.560102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:03.239993 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:03.239963 2577 scope.go:117] "RemoveContainer" containerID="0d8f3afa46d0c361e0d2c181efbd9c61b287e0d67d6c7c7f518b52a6e3a64a34" Jul 10 21:09:03.240331 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:09:03.240212 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz_odh-ai-gateway-infra(9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" Jul 10 21:09:05.250514 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:05.250476 2577 generic.go:356] "Generic (PLEG): container finished" podID="11cd7a03-1814-4249-87f8-df8feac03c57" containerID="7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62" exitCode=0 Jul 10 21:09:05.251037 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:05.250538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" event={"ID":"11cd7a03-1814-4249-87f8-df8feac03c57","Type":"ContainerDied","Data":"7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62"} Jul 10 21:09:05.251037 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:05.250922 2577 scope.go:117] "RemoveContainer" containerID="7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62" Jul 10 21:09:05.251903 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:05.251871 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:09:05.251965 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:05.251947 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:09:05.252005 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:05.251981 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:09:06.259628 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:06.259591 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" event={"ID":"11cd7a03-1814-4249-87f8-df8feac03c57","Type":"ContainerStarted","Data":"ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa"} Jul 10 21:09:06.491377 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:06.491344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:09:06.492565 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:06.492539 2577 patch_prober.go:28] interesting pod/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 10 21:09:06.492722 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:06.492587 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 10 21:09:07.492298 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:07.492221 2577 patch_prober.go:28] interesting pod/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 10 21:09:07.492298 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:07.492278 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 10 21:09:08.491976 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:08.491939 2577 patch_prober.go:28] interesting pod/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 10 21:09:08.492145 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:08.491996 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 10 21:09:09.491619 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:09.491583 2577 patch_prober.go:28] interesting pod/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 10 21:09:09.492021 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:09.491644 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 10 21:09:10.191707 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:10.191636 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:09:10.495105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:10.495076 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:09:10.496037 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:10.496017 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:09:12.210467 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:12.210433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:15.164527 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:15.164494 2577 scope.go:117] "RemoveContainer" containerID="0d8f3afa46d0c361e0d2c181efbd9c61b287e0d67d6c7c7f518b52a6e3a64a34" Jul 10 21:09:16.307332 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:16.307294 2577 generic.go:356] "Generic (PLEG): container finished" podID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerID="bda5b04a41d262a3ab5b6ac47a8977b30b3be919465a5a1f1c86927a84835e25" exitCode=0 Jul 10 21:09:16.307803 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:16.307367 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" event={"ID":"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d","Type":"ContainerDied","Data":"bda5b04a41d262a3ab5b6ac47a8977b30b3be919465a5a1f1c86927a84835e25"} Jul 10 21:09:16.307803 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:16.307457 2577 scope.go:117] "RemoveContainer" containerID="0d8f3afa46d0c361e0d2c181efbd9c61b287e0d67d6c7c7f518b52a6e3a64a34" Jul 10 21:09:17.335649 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:17.335613 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz"] Jul 10 21:09:17.450843 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:17.450819 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:09:17.539143 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:17.539112 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d-maas-api-service-ca\") pod \"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d\" (UID: \"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d\") " Jul 10 21:09:17.539489 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:17.539469 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" (UID: "9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:09:17.640730 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:17.640629 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:09:18.319533 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:18.319501 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" event={"ID":"9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d","Type":"ContainerDied","Data":"ff4e6d2f5d1e7cb0be1a70ed3043cc1949912ed2caef004f99a63a5977991c83"} Jul 10 21:09:18.319738 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:18.319548 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff4e6d2f5d1e7cb0be1a70ed3043cc1949912ed2caef004f99a63a5977991c83" Jul 10 21:09:18.319738 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:18.319513 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz" Jul 10 21:09:18.348770 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:18.348734 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz"] Jul 10 21:09:18.353568 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:18.353545 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6fde404a-69jhz"] Jul 10 21:09:19.164439 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:19.164405 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" path="/var/lib/kubelet/pods/9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d/volumes" Jul 10 21:09:59.008110 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:59.008071 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp"] Jul 10 21:09:59.008543 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:59.008296 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" containerID="cri-o://ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa" gracePeriod=30 Jul 10 21:09:59.033212 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:59.033185 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:59.141483 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:59.141446 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:59.435473 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:09:59.435374 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:01.954408 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:01.954373 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:10:04.266103 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.266076 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:10:04.363457 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363433 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-certs\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363578 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363461 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-data\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363578 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363505 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11cd7a03-1814-4249-87f8-df8feac03c57-istiod-ca-cert\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363578 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363547 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-envoy\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363583 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-credential-socket\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363613 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11cd7a03-1814-4249-87f8-df8feac03c57-istio-podinfo\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363644 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2sqt\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-kube-api-access-q2sqt\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363683 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-socket\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.363761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363699 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:04.363761 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363746 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-istio-token\") pod \"11cd7a03-1814-4249-87f8-df8feac03c57\" (UID: \"11cd7a03-1814-4249-87f8-df8feac03c57\") " Jul 10 21:10:04.364083 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363911 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:04.364083 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363925 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:04.364083 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.363968 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11cd7a03-1814-4249-87f8-df8feac03c57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:10:04.364236 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.364116 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.364236 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.364137 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11cd7a03-1814-4249-87f8-df8feac03c57-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.364236 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.364151 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.364236 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.364165 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.364376 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.364352 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-data" (OuterVolumeSpecName: "istio-data") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:04.365851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.365830 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11cd7a03-1814-4249-87f8-df8feac03c57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:10:04.365963 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.365835 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-istio-token" (OuterVolumeSpecName: "istio-token") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:10:04.366124 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.366106 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:04.366185 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.366142 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-kube-api-access-q2sqt" (OuterVolumeSpecName: "kube-api-access-q2sqt") pod "11cd7a03-1814-4249-87f8-df8feac03c57" (UID: "11cd7a03-1814-4249-87f8-df8feac03c57"). InnerVolumeSpecName "kube-api-access-q2sqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:10:04.464795 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.464755 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.464795 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.464788 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11cd7a03-1814-4249-87f8-df8feac03c57-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.464795 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.464797 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11cd7a03-1814-4249-87f8-df8feac03c57-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.464795 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.464806 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2sqt\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-kube-api-access-q2sqt\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.465043 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.464816 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11cd7a03-1814-4249-87f8-df8feac03c57-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:10:04.534382 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.534351 2577 generic.go:356] "Generic (PLEG): container finished" podID="11cd7a03-1814-4249-87f8-df8feac03c57" containerID="ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa" exitCode=0 Jul 10 21:10:04.534553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.534399 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" Jul 10 21:10:04.534553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.534426 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" event={"ID":"11cd7a03-1814-4249-87f8-df8feac03c57","Type":"ContainerDied","Data":"ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa"} Jul 10 21:10:04.534553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.534491 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp" event={"ID":"11cd7a03-1814-4249-87f8-df8feac03c57","Type":"ContainerDied","Data":"b601d5375c88de9fc7f9dca85447dad9ab4a24951ec80c86a29280685a09aa11"} Jul 10 21:10:04.534553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.534523 2577 scope.go:117] "RemoveContainer" containerID="ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa" Jul 10 21:10:04.543734 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.543711 2577 scope.go:117] "RemoveContainer" containerID="7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62" Jul 10 21:10:04.553863 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.553839 2577 scope.go:117] "RemoveContainer" containerID="ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa" Jul 10 21:10:04.554601 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:10:04.554545 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa\": container with ID starting with ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa not found: ID does not exist" containerID="ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa" Jul 10 21:10:04.554601 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.554586 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa"} err="failed to get container status \"ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa\": rpc error: code = NotFound desc = could not find container \"ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa\": container with ID starting with ae5533d8884f2dd7be38b1fc8bb3858d26262525ca2f87c7f0e8bbf545868daa not found: ID does not exist" Jul 10 21:10:04.554786 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.554610 2577 scope.go:117] "RemoveContainer" containerID="7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62" Jul 10 21:10:04.554915 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:10:04.554894 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62\": container with ID starting with 7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62 not found: ID does not exist" containerID="7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62" Jul 10 21:10:04.554961 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.554921 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62"} err="failed to get container status \"7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62\": rpc error: code = NotFound desc = could not find container \"7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62\": container with ID starting with 7a3046e50aef93bc3a9d4c80f1f00b75b2cc3a4bc948e0e96525836dde572b62 not found: ID does not exist" Jul 10 21:10:04.555898 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.555879 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp"] Jul 10 21:10:04.559263 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.559242 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6fde404a-gw-openshift-default-664b8cdbff-rj4zp"] Jul 10 21:10:04.844200 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:04.844166 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:05.070943 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.070910 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7"] Jul 10 21:10:05.071630 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071603 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.071796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071686 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.071796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071708 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" Jul 10 21:10:05.071796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071729 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" Jul 10 21:10:05.071796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071765 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.071796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071772 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.072045 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071869 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" Jul 10 21:10:05.072045 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071886 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" Jul 10 21:10:05.072045 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071896 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.072045 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071906 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.072045 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.071916 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:10:05.076465 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.076440 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.078794 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.078772 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6bb25bb7-openshift-default-dockercfg-vbxjh\"" Jul 10 21:10:05.087272 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.087244 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7"] Jul 10 21:10:05.108948 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.108857 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:05.170158 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.170126 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" path="/var/lib/kubelet/pods/11cd7a03-1814-4249-87f8-df8feac03c57/volumes" Jul 10 21:10:05.172256 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptn5t\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-kube-api-access-ptn5t\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172358 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-certs\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172358 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-socket\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172358 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-token\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172358 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istiod-ca-cert\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-data\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-podinfo\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-envoy\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.172567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.172541 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-credential-socket\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.209596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.209562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:05.273553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-certs\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.273553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-socket\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-token\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istiod-ca-cert\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-data\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-podinfo\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-envoy\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-credential-socket\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273968 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptn5t\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-kube-api-access-ptn5t\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.273968 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-socket\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.274092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-certs\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274493 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.274260 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-data\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274493 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.274284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-credential-socket\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.274493 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.274471 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istiod-ca-cert\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.276265 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.276238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-envoy\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.276480 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.276461 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-podinfo\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.281468 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.281446 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-token\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.281784 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.281764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptn5t\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-kube-api-access-ptn5t\") pod \"e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.389032 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.388937 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:05.526165 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.526137 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7"] Jul 10 21:10:05.527990 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:10:05.527962 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca4b72b_cb0a_4865_84e5_fdd03f51e0c5.slice/crio-d11447fd5899d7e2e14eaf234414c40ae040dd67a3093ffc59da11b722692972 WatchSource:0}: Error finding container d11447fd5899d7e2e14eaf234414c40ae040dd67a3093ffc59da11b722692972: Status 404 returned error can't find the container with id d11447fd5899d7e2e14eaf234414c40ae040dd67a3093ffc59da11b722692972 Jul 10 21:10:05.530300 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.530260 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:10:05.530388 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.530347 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:10:05.530450 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.530397 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:10:05.540870 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.540845 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" event={"ID":"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5","Type":"ContainerStarted","Data":"d11447fd5899d7e2e14eaf234414c40ae040dd67a3093ffc59da11b722692972"} Jul 10 21:10:05.647985 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.647900 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:05.818625 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:05.818580 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:06.294004 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.293960 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:06.387502 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.387469 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:06.505234 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.505203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:06.549835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.549737 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" event={"ID":"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5","Type":"ContainerStarted","Data":"cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c"} Jul 10 21:10:06.572703 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.572612 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" podStartSLOduration=1.572589846 podStartE2EDuration="1.572589846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:10:06.569950831 +0000 UTC m=+1941.918260186" watchObservedRunningTime="2026-07-10 21:10:06.572589846 +0000 UTC m=+1941.920899178" Jul 10 21:10:06.639093 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.639060 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:06.981496 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:06.981412 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:07.083026 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:07.082998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:07.389580 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:07.389545 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:07.394356 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:07.394326 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:07.554044 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:07.554014 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:07.555051 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:07.555035 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:09.172878 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:09.172847 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:13.346217 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:13.346184 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:10:21.630152 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:21.630074 2577 generic.go:356] "Generic (PLEG): container finished" podID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerID="cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c" exitCode=0 Jul 10 21:10:21.630566 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:21.630148 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" event={"ID":"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5","Type":"ContainerDied","Data":"cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c"} Jul 10 21:10:21.630610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:21.630580 2577 scope.go:117] "RemoveContainer" containerID="cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c" Jul 10 21:10:21.631595 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:21.631569 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:10:21.631646 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:21.631632 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:10:21.631715 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:21.631680 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:10:22.554273 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:22.554220 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:22.640343 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:22.640295 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" event={"ID":"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5","Type":"ContainerStarted","Data":"428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d"} Jul 10 21:10:23.390140 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:23.390106 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:23.391216 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:23.391187 2577 patch_prober.go:28] interesting pod/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 21:10:23.391324 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:23.391246 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 21:10:24.389862 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:24.389831 2577 patch_prober.go:28] interesting pod/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 21:10:24.390211 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:24.389882 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 21:10:25.390034 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:25.390001 2577 patch_prober.go:28] interesting pod/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 10 21:10:25.390395 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:25.390054 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 10 21:10:26.393788 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:26.393761 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:26.394194 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:26.394116 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:26.394867 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:26.394848 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:10:49.002344 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:49.002307 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:49.103738 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:10:49.103697 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:11.324944 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.324909 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk"] Jul 10 21:11:11.325328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.325259 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" Jul 10 21:11:11.325328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.325275 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cd7a03-1814-4249-87f8-df8feac03c57" containerName="istio-proxy" Jul 10 21:11:11.325328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.325296 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:11:11.325328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.325301 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f4f42fb-cd84-4c22-85c6-1eb19eef7a2d" containerName="revoke-keys" Jul 10 21:11:11.328575 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.328553 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:11.330846 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.330826 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:11:11.330940 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.330884 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:11:11.334868 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.334845 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk"] Jul 10 21:11:11.367147 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.367116 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f210d3b-dbbe-4709-9321-3e42dd8f4c12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk\" (UID: \"6f210d3b-dbbe-4709-9321-3e42dd8f4c12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:11.468628 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.468586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f210d3b-dbbe-4709-9321-3e42dd8f4c12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk\" (UID: \"6f210d3b-dbbe-4709-9321-3e42dd8f4c12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:11.469242 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.469220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f210d3b-dbbe-4709-9321-3e42dd8f4c12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk\" (UID: \"6f210d3b-dbbe-4709-9321-3e42dd8f4c12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:11.641277 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.641188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:11.774769 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.774741 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk"] Jul 10 21:11:11.777514 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:11:11.777488 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f210d3b_dbbe_4709_9321_3e42dd8f4c12.slice/crio-5fa8283ef190af51e44c62a2ac5d0251d180bac26687d5e3e2607f0959ee7b69 WatchSource:0}: Error finding container 5fa8283ef190af51e44c62a2ac5d0251d180bac26687d5e3e2607f0959ee7b69: Status 404 returned error can't find the container with id 5fa8283ef190af51e44c62a2ac5d0251d180bac26687d5e3e2607f0959ee7b69 Jul 10 21:11:11.880063 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.880027 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" event={"ID":"6f210d3b-dbbe-4709-9321-3e42dd8f4c12","Type":"ContainerStarted","Data":"5fa8283ef190af51e44c62a2ac5d0251d180bac26687d5e3e2607f0959ee7b69"} Jul 10 21:11:11.997099 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:11.997064 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:12.097640 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:12.097601 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:12.887460 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:12.887409 2577 generic.go:356] "Generic (PLEG): container finished" podID="6f210d3b-dbbe-4709-9321-3e42dd8f4c12" containerID="979c068d8d9e51b2add3cf00e02f6d52e0ff07a977b69df2d646e1add5da25aa" exitCode=0 Jul 10 21:11:12.887873 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:12.887454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" event={"ID":"6f210d3b-dbbe-4709-9321-3e42dd8f4c12","Type":"ContainerDied","Data":"979c068d8d9e51b2add3cf00e02f6d52e0ff07a977b69df2d646e1add5da25aa"} Jul 10 21:11:14.025604 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.025578 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:14.090372 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.090347 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f210d3b-dbbe-4709-9321-3e42dd8f4c12-maas-api-service-ca\") pod \"6f210d3b-dbbe-4709-9321-3e42dd8f4c12\" (UID: \"6f210d3b-dbbe-4709-9321-3e42dd8f4c12\") " Jul 10 21:11:14.090726 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.090702 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f210d3b-dbbe-4709-9321-3e42dd8f4c12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f210d3b-dbbe-4709-9321-3e42dd8f4c12" (UID: "6f210d3b-dbbe-4709-9321-3e42dd8f4c12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:11:14.191056 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.190974 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f210d3b-dbbe-4709-9321-3e42dd8f4c12-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:14.898480 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.898455 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" Jul 10 21:11:14.898624 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.898451 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk" event={"ID":"6f210d3b-dbbe-4709-9321-3e42dd8f4c12","Type":"ContainerDied","Data":"5fa8283ef190af51e44c62a2ac5d0251d180bac26687d5e3e2607f0959ee7b69"} Jul 10 21:11:14.898624 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:14.898577 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fa8283ef190af51e44c62a2ac5d0251d180bac26687d5e3e2607f0959ee7b69" Jul 10 21:11:21.820142 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:21.820109 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:24.948321 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:24.948289 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:11:26.411392 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:26.411357 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:32.239772 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:32.239731 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:32.340639 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:32.340612 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:37.406122 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.406090 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7"] Jul 10 21:11:37.456651 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.456621 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:37.698796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.698711 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6"] Jul 10 21:11:37.699160 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.699130 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f210d3b-dbbe-4709-9321-3e42dd8f4c12" containerName="revoke-keys" Jul 10 21:11:37.699160 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.699152 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f210d3b-dbbe-4709-9321-3e42dd8f4c12" containerName="revoke-keys" Jul 10 21:11:37.699367 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.699244 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f210d3b-dbbe-4709-9321-3e42dd8f4c12" containerName="revoke-keys" Jul 10 21:11:37.702457 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.702436 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.703896 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.703871 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-socket\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704039 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.703926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istiod-ca-cert\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704106 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-credential-socket\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704106 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-certs\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704214 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704124 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-token\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704214 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-envoy\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704312 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-data\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704312 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-podinfo\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.704312 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.704289 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5jdb\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-kube-api-access-k5jdb\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.705105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.705076 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-183404a5-openshift-default-dockercfg-w6jfg\"" Jul 10 21:11:37.713733 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.713712 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6"] Jul 10 21:11:37.738831 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.738804 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:37.804832 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.804803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-envoy\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.804954 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.804846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-data\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.804954 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.804888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-podinfo\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.804954 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.804912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5jdb\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-kube-api-access-k5jdb\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.804954 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.804945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-socket\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805179 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.804968 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istiod-ca-cert\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805179 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.805013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-credential-socket\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805179 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.805064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-certs\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805179 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.805086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-token\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805808 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.805499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-socket\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805923 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.805832 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-credential-socket\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.805994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.805971 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-data\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.806139 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.806119 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-certs\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.806252 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.806227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istiod-ca-cert\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.807357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.807336 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-envoy\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.808591 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.808570 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-podinfo\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.819799 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.819740 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5jdb\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-kube-api-access-k5jdb\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:37.820145 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:37.820121 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-token\") pod \"e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:38.004625 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.004569 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" containerID="cri-o://428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d" gracePeriod=30 Jul 10 21:11:38.013861 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.013839 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:38.149876 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.149843 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6"] Jul 10 21:11:38.152354 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:11:38.152328 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dbe4768_01cf_4dbc_bdb6_64924336fdf4.slice/crio-8064df67c165e39178bd301678454834acb72a948e86548c76266cfccaea8afe WatchSource:0}: Error finding container 8064df67c165e39178bd301678454834acb72a948e86548c76266cfccaea8afe: Status 404 returned error can't find the container with id 8064df67c165e39178bd301678454834acb72a948e86548c76266cfccaea8afe Jul 10 21:11:38.154919 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.154885 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:11:38.155029 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.154949 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:11:38.155029 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.154978 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:11:38.858107 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.858077 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:38.944957 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:38.944915 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.012554 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.012522 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" event={"ID":"2dbe4768-01cf-4dbc-bdb6-64924336fdf4","Type":"ContainerStarted","Data":"f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34"} Jul 10 21:11:39.012725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.012582 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" event={"ID":"2dbe4768-01cf-4dbc-bdb6-64924336fdf4","Type":"ContainerStarted","Data":"8064df67c165e39178bd301678454834acb72a948e86548c76266cfccaea8afe"} Jul 10 21:11:39.014135 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.014065 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:39.014515 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.014496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.019551 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.019531 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:39.033241 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.033197 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" podStartSLOduration=2.033182507 podStartE2EDuration="2.033182507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:11:39.031945013 +0000 UTC m=+2034.380254343" watchObservedRunningTime="2026-07-10 21:11:39.033182507 +0000 UTC m=+2034.381491862" Jul 10 21:11:39.115200 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.115116 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.580862 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.580836 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.673972 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:39.673942 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:40.018282 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:40.018247 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:40.019186 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:40.019167 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:41.396721 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:41.396681 2577 patch_prober.go:28] interesting pod/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:11:41.397129 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:41.396750 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:11:41.649663 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:41.649568 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:11:43.307822 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.307797 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:11:43.359679 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.359321 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-credential-socket\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.359679 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.359552 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.359912 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.359704 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptn5t\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-kube-api-access-ptn5t\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.360097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.360077 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-envoy\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.360547 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.360529 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-socket\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.360722 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.360703 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-token\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.360952 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.360937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-data\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.361075 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361061 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istiod-ca-cert\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.361203 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361190 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-certs\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.361328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361314 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-podinfo\") pod \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\" (UID: \"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5\") " Jul 10 21:11:43.361758 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361738 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.361862 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.360844 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.361924 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361379 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:11:43.361924 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361588 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.362043 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.361922 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.363071 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.363037 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.364136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.364102 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:43.364692 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.364642 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-kube-api-access-ptn5t" (OuterVolumeSpecName: "kube-api-access-ptn5t") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "kube-api-access-ptn5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:43.365702 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.365675 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" (UID: "fca4b72b-cb0a-4865-84e5-fdd03f51e0c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:11:43.462539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462510 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptn5t\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-kube-api-access-ptn5t\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462537 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462546 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462556 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462564 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462572 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462580 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.462774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:43.462587 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:11:44.044366 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.044321 2577 generic.go:356] "Generic (PLEG): container finished" podID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerID="428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d" exitCode=0 Jul 10 21:11:44.044581 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.044385 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" event={"ID":"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5","Type":"ContainerDied","Data":"428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d"} Jul 10 21:11:44.044581 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.044439 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" event={"ID":"fca4b72b-cb0a-4865-84e5-fdd03f51e0c5","Type":"ContainerDied","Data":"d11447fd5899d7e2e14eaf234414c40ae040dd67a3093ffc59da11b722692972"} Jul 10 21:11:44.044581 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.044470 2577 scope.go:117] "RemoveContainer" containerID="428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d" Jul 10 21:11:44.044813 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.044636 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7" Jul 10 21:11:44.063423 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.063405 2577 scope.go:117] "RemoveContainer" containerID="cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c" Jul 10 21:11:44.071027 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.071006 2577 scope.go:117] "RemoveContainer" containerID="428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d" Jul 10 21:11:44.071271 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:11:44.071252 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d\": container with ID starting with 428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d not found: ID does not exist" containerID="428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d" Jul 10 21:11:44.071318 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.071280 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d"} err="failed to get container status \"428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d\": rpc error: code = NotFound desc = could not find container \"428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d\": container with ID starting with 428629b4e849412d66079a1c0f7884f28a669c40e44be3730e7d3b5a2551605d not found: ID does not exist" Jul 10 21:11:44.071318 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.071298 2577 scope.go:117] "RemoveContainer" containerID="cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c" Jul 10 21:11:44.071516 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:11:44.071501 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c\": container with ID starting with cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c not found: ID does not exist" containerID="cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c" Jul 10 21:11:44.071560 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.071521 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c"} err="failed to get container status \"cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c\": rpc error: code = NotFound desc = could not find container \"cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c\": container with ID starting with cd994fb2b6d510caf39984fcb6c3fa5d0c59d29d9fa9c3597b4546e49e5ab61c not found: ID does not exist" Jul 10 21:11:44.074637 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.074615 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7"] Jul 10 21:11:44.078568 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:44.078543 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6bb25bb7-openshift-default-5496f7d67-x5rd7"] Jul 10 21:11:45.168864 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:45.167045 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" path="/var/lib/kubelet/pods/fca4b72b-cb0a-4865-84e5-fdd03f51e0c5/volumes" Jul 10 21:11:49.043777 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:49.043744 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:49.108540 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:49.108507 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:50.079997 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:50.079966 2577 generic.go:356] "Generic (PLEG): container finished" podID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerID="f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34" exitCode=0 Jul 10 21:11:50.080422 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:50.080039 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" event={"ID":"2dbe4768-01cf-4dbc-bdb6-64924336fdf4","Type":"ContainerDied","Data":"f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34"} Jul 10 21:11:50.080422 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:50.080409 2577 scope.go:117] "RemoveContainer" containerID="f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34" Jul 10 21:11:50.081362 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:50.081336 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:11:50.081417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:50.081401 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:11:50.081463 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:50.081438 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:11:51.087772 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:51.087732 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" event={"ID":"2dbe4768-01cf-4dbc-bdb6-64924336fdf4","Type":"ContainerStarted","Data":"80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065"} Jul 10 21:11:52.015047 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:52.015011 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:52.016187 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:52.016160 2577 patch_prober.go:28] interesting pod/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 10 21:11:52.016332 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:52.016232 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 10 21:11:53.014549 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:53.014509 2577 patch_prober.go:28] interesting pod/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 10 21:11:53.015060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:53.014566 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 10 21:11:53.159679 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:53.158503 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:54.014470 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:54.014427 2577 patch_prober.go:28] interesting pod/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 10 21:11:54.014692 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:54.014500 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 10 21:11:55.018231 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:55.018202 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:55.018614 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:55.018311 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:11:55.019406 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:11:55.019381 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:12:20.526980 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:20.526905 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:33.293161 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.293125 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9"] Jul 10 21:12:33.293603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.293433 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" Jul 10 21:12:33.293603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.293444 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" Jul 10 21:12:33.293603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.293516 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" Jul 10 21:12:33.296557 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.296539 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:33.299151 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.299130 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:12:33.299151 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.299150 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:12:33.304523 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.304503 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9"] Jul 10 21:12:33.436889 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.436858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f293be-0034-4e22-b9c4-3547a423c803-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-183404a5-2cjm9\" (UID: \"78f293be-0034-4e22-b9c4-3547a423c803\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:33.538256 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.538226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f293be-0034-4e22-b9c4-3547a423c803-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-183404a5-2cjm9\" (UID: \"78f293be-0034-4e22-b9c4-3547a423c803\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:33.538878 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.538859 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f293be-0034-4e22-b9c4-3547a423c803-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-183404a5-2cjm9\" (UID: \"78f293be-0034-4e22-b9c4-3547a423c803\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:33.608334 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.608272 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:33.736060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:33.736029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9"] Jul 10 21:12:33.737330 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:12:33.737291 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f293be_0034_4e22_b9c4_3547a423c803.slice/crio-e65ce17725980e0342727cd71f48266aead57303456557b2908d1e7a33c73b2b WatchSource:0}: Error finding container e65ce17725980e0342727cd71f48266aead57303456557b2908d1e7a33c73b2b: Status 404 returned error can't find the container with id e65ce17725980e0342727cd71f48266aead57303456557b2908d1e7a33c73b2b Jul 10 21:12:34.295941 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:34.295909 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" event={"ID":"78f293be-0034-4e22-b9c4-3547a423c803","Type":"ContainerStarted","Data":"dc96c6a569adc987f1e7246fa169917182318879bc7a17097af689583a05f9ed"} Jul 10 21:12:34.296320 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:34.295969 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" event={"ID":"78f293be-0034-4e22-b9c4-3547a423c803","Type":"ContainerStarted","Data":"e65ce17725980e0342727cd71f48266aead57303456557b2908d1e7a33c73b2b"} Jul 10 21:12:34.312411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:34.312361 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" podStartSLOduration=1.312345208 podStartE2EDuration="1.312345208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:12:34.311695671 +0000 UTC m=+2089.660005001" watchObservedRunningTime="2026-07-10 21:12:34.312345208 +0000 UTC m=+2089.660654537" Jul 10 21:12:35.301294 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:35.301254 2577 generic.go:356] "Generic (PLEG): container finished" podID="78f293be-0034-4e22-b9c4-3547a423c803" containerID="dc96c6a569adc987f1e7246fa169917182318879bc7a17097af689583a05f9ed" exitCode=0 Jul 10 21:12:35.301683 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:35.301339 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" event={"ID":"78f293be-0034-4e22-b9c4-3547a423c803","Type":"ContainerDied","Data":"dc96c6a569adc987f1e7246fa169917182318879bc7a17097af689583a05f9ed"} Jul 10 21:12:36.435765 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:36.435743 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:36.565166 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:36.565085 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f293be-0034-4e22-b9c4-3547a423c803-maas-api-service-ca\") pod \"78f293be-0034-4e22-b9c4-3547a423c803\" (UID: \"78f293be-0034-4e22-b9c4-3547a423c803\") " Jul 10 21:12:36.565402 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:36.565380 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f293be-0034-4e22-b9c4-3547a423c803-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78f293be-0034-4e22-b9c4-3547a423c803" (UID: "78f293be-0034-4e22-b9c4-3547a423c803"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:12:36.666459 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:36.666431 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78f293be-0034-4e22-b9c4-3547a423c803-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:12:37.311795 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:37.311760 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" Jul 10 21:12:37.311943 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:37.311768 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-183404a5-2cjm9" event={"ID":"78f293be-0034-4e22-b9c4-3547a423c803","Type":"ContainerDied","Data":"e65ce17725980e0342727cd71f48266aead57303456557b2908d1e7a33c73b2b"} Jul 10 21:12:37.311943 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:37.311885 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e65ce17725980e0342727cd71f48266aead57303456557b2908d1e7a33c73b2b" Jul 10 21:12:42.068110 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:42.068078 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:44.103211 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:44.103177 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:47.253237 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:47.253204 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:12:48.378256 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:48.378217 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:54.128646 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:54.128611 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:54.188243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:54.188218 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:58.558819 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:58.558769 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6"] Jul 10 21:12:58.559209 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:58.558968 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" containerID="cri-o://80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065" gracePeriod=30 Jul 10 21:12:58.585026 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:58.584997 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:59.168195 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:59.168163 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:59.268242 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:59.268209 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:59.367369 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:12:59.367340 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:00.859435 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:00.859404 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:03.813468 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.813445 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:13:03.901583 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901555 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-token\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.901740 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901601 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-certs\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.901740 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901639 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-credential-socket\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.901740 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901677 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-envoy\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.901909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901743 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-podinfo\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.901909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901777 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-socket\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.901909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-data\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.902060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901960 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:13:03.902060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901975 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:13:03.902060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.901991 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istiod-ca-cert\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.902060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.902031 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5jdb\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-kube-api-access-k5jdb\") pod \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\" (UID: \"2dbe4768-01cf-4dbc-bdb6-64924336fdf4\") " Jul 10 21:13:03.902355 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.902336 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:03.902391 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.902363 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:03.902773 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.902719 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:13:03.902773 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.902748 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:13:03.903313 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.903283 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-data" (OuterVolumeSpecName: "istio-data") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:13:03.903924 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.903897 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-token" (OuterVolumeSpecName: "istio-token") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:13:03.904467 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.904436 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:13:03.904467 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.904453 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-kube-api-access-k5jdb" (OuterVolumeSpecName: "kube-api-access-k5jdb") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "kube-api-access-k5jdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:13:03.904818 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:03.904791 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2dbe4768-01cf-4dbc-bdb6-64924336fdf4" (UID: "2dbe4768-01cf-4dbc-bdb6-64924336fdf4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:13:04.003139 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003107 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.003139 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003134 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.003139 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003144 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.003417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003153 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.003417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003162 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.003417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003171 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.003417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.003180 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5jdb\" (UniqueName: \"kubernetes.io/projected/2dbe4768-01cf-4dbc-bdb6-64924336fdf4-kube-api-access-k5jdb\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:13:04.440775 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.440746 2577 generic.go:356] "Generic (PLEG): container finished" podID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerID="80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065" exitCode=0 Jul 10 21:13:04.441111 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.440783 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" event={"ID":"2dbe4768-01cf-4dbc-bdb6-64924336fdf4","Type":"ContainerDied","Data":"80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065"} Jul 10 21:13:04.441111 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.440792 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" Jul 10 21:13:04.441111 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.440826 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6" event={"ID":"2dbe4768-01cf-4dbc-bdb6-64924336fdf4","Type":"ContainerDied","Data":"8064df67c165e39178bd301678454834acb72a948e86548c76266cfccaea8afe"} Jul 10 21:13:04.441111 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.440847 2577 scope.go:117] "RemoveContainer" containerID="80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065" Jul 10 21:13:04.450453 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.450436 2577 scope.go:117] "RemoveContainer" containerID="f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34" Jul 10 21:13:04.458171 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.458153 2577 scope.go:117] "RemoveContainer" containerID="80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065" Jul 10 21:13:04.458412 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:13:04.458394 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065\": container with ID starting with 80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065 not found: ID does not exist" containerID="80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065" Jul 10 21:13:04.458461 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.458423 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065"} err="failed to get container status \"80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065\": rpc error: code = NotFound desc = could not find container \"80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065\": container with ID starting with 80d0b7cf59d4b34f8e04754c3cae5bc2350319010e0771428fac3bbb185e2065 not found: ID does not exist" Jul 10 21:13:04.458461 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.458440 2577 scope.go:117] "RemoveContainer" containerID="f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34" Jul 10 21:13:04.458676 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:13:04.458638 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34\": container with ID starting with f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34 not found: ID does not exist" containerID="f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34" Jul 10 21:13:04.458730 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.458679 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34"} err="failed to get container status \"f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34\": rpc error: code = NotFound desc = could not find container \"f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34\": container with ID starting with f49fbcd8dcf4b1363029b1626281ca4290047c1a3664e3ae20e23a1d5f032f34 not found: ID does not exist" Jul 10 21:13:04.464807 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.464785 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6"] Jul 10 21:13:04.467216 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:04.467194 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-183404a5-openshift-default-df6669f7f-g2zz6"] Jul 10 21:13:05.165732 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:05.165698 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" path="/var/lib/kubelet/pods/2dbe4768-01cf-4dbc-bdb6-64924336fdf4/volumes" Jul 10 21:13:05.514837 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:05.514805 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:06.755017 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:06.754987 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:06.860710 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:06.860677 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:06.956423 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:06.956393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:07.032205 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:07.032125 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:07.464373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:07.464270 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:33.449516 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:33.449459 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:39.551878 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:39.551780 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:39.615396 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:39.615368 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:39.710345 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:39.710312 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:39.805126 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:39.805046 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:13:55.667957 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:13:55.667925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:02.372301 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:02.372266 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:02.491703 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:02.491673 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:02.608492 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:02.608459 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:13.144080 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:13.144047 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:13.926624 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:13.926591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:19.791836 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:19.791803 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:19.870463 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:19.870433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:19.971968 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:19.971915 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:20.084143 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:20.084055 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:20.928992 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:20.928963 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:21.030067 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:21.030036 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:21.129706 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:21.129679 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:22.156193 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:22.156157 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:14:22.759239 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:22.759210 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:28.647212 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:28.647179 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:28.760898 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:28.760864 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:29.346106 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:29.346079 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:31.357267 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:31.357225 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:14:35.047336 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.047303 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:35.147920 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.147889 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:35.282693 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.282637 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:35.575304 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575269 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f"] Jul 10 21:14:35.575780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575765 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f293be-0034-4e22-b9c4-3547a423c803" containerName="revoke-keys" Jul 10 21:14:35.575848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575784 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f293be-0034-4e22-b9c4-3547a423c803" containerName="revoke-keys" Jul 10 21:14:35.575848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575800 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" Jul 10 21:14:35.575848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575810 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" Jul 10 21:14:35.575848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575830 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" Jul 10 21:14:35.575848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575840 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" Jul 10 21:14:35.576012 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575851 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" Jul 10 21:14:35.576012 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575860 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" Jul 10 21:14:35.576012 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575937 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" Jul 10 21:14:35.576012 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575948 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dbe4768-01cf-4dbc-bdb6-64924336fdf4" containerName="istio-proxy" Jul 10 21:14:35.576012 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575956 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f293be-0034-4e22-b9c4-3547a423c803" containerName="revoke-keys" Jul 10 21:14:35.576012 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.575963 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca4b72b-cb0a-4865-84e5-fdd03f51e0c5" containerName="istio-proxy" Jul 10 21:14:35.579081 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.579064 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.581416 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.581392 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dd381648-openshift-default-dockercfg-dj6st\"" Jul 10 21:14:35.605519 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.600625 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f"] Jul 10 21:14:35.608066 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.608041 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:35.608867 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.608837 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-istio-token\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.608977 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.608883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5f61c5b-9c81-4104-8689-442699160eb1-istio-podinfo\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.608977 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.608908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-socket\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.609098 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.609008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-data\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.609098 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.609046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6rpx\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-kube-api-access-j6rpx\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.609098 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.609080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5f61c5b-9c81-4104-8689-442699160eb1-istiod-ca-cert\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.609258 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.609148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-certs\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.609258 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.609195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-envoy\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.609258 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.609249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-credential-socket\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709673 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-envoy\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-credential-socket\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-istio-token\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709749 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5f61c5b-9c81-4104-8689-442699160eb1-istio-podinfo\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709784 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-socket\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-data\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.709885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709855 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6rpx\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-kube-api-access-j6rpx\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710229 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5f61c5b-9c81-4104-8689-442699160eb1-istiod-ca-cert\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710229 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.709947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-certs\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710229 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.710172 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-credential-socket\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710367 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.710238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-socket\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710412 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.710382 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-data\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710471 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.710448 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-certs\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.710785 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.710769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5f61c5b-9c81-4104-8689-442699160eb1-istiod-ca-cert\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.712633 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.712612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-envoy\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.712780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.712763 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5f61c5b-9c81-4104-8689-442699160eb1-istio-podinfo\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.718482 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.718460 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-istio-token\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.718727 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.718704 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6rpx\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-kube-api-access-j6rpx\") pod \"e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:35.782429 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.782388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:35.889742 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:35.889644 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:36.025014 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.024986 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f"] Jul 10 21:14:36.027545 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:14:36.027518 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f61c5b_9c81_4104_8689_442699160eb1.slice/crio-5f4d6555b075b3c0f5f5cca60730bb3c135c308fc84305ca0c33273059b1a11d WatchSource:0}: Error finding container 5f4d6555b075b3c0f5f5cca60730bb3c135c308fc84305ca0c33273059b1a11d: Status 404 returned error can't find the container with id 5f4d6555b075b3c0f5f5cca60730bb3c135c308fc84305ca0c33273059b1a11d Jul 10 21:14:36.029583 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.029549 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:14:36.029690 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.029618 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:14:36.029690 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.029646 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:14:36.881888 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.881849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" event={"ID":"b5f61c5b-9c81-4104-8689-442699160eb1","Type":"ContainerStarted","Data":"fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be"} Jul 10 21:14:36.882292 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.881914 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" event={"ID":"b5f61c5b-9c81-4104-8689-442699160eb1","Type":"ContainerStarted","Data":"5f4d6555b075b3c0f5f5cca60730bb3c135c308fc84305ca0c33273059b1a11d"} Jul 10 21:14:36.890161 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.890134 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:36.904740 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:36.904687 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" podStartSLOduration=1.904647458 podStartE2EDuration="1.904647458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:14:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:14:36.903378911 +0000 UTC m=+2212.251688240" watchObservedRunningTime="2026-07-10 21:14:36.904647458 +0000 UTC m=+2212.252956788" Jul 10 21:14:37.895743 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:37.895712 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:38.836787 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:38.836754 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:38.892377 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:38.892346 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:38.893400 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:38.893378 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:38.930602 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:38.930568 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:39.038999 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:39.038963 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:39.278522 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:39.278482 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:39.682920 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:39.682808 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:39.782767 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:39.782732 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:43.338753 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:43.338707 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:44.005380 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:44.005342 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:44.095819 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:44.095784 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:44.158521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:44.158489 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:14:44.444030 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:44.443943 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:45.264705 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:45.264649 2577 scope.go:117] "RemoveContainer" containerID="f00dfa2bd0a8034e64d398efe5ca328a735a211833b043aa84a12c207ff0b936" Jul 10 21:14:50.681915 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:50.681877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:50.779778 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:50.779744 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:52.967770 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:52.967738 2577 generic.go:356] "Generic (PLEG): container finished" podID="b5f61c5b-9c81-4104-8689-442699160eb1" containerID="fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be" exitCode=0 Jul 10 21:14:52.968167 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:52.967781 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" event={"ID":"b5f61c5b-9c81-4104-8689-442699160eb1","Type":"ContainerDied","Data":"fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be"} Jul 10 21:14:52.968167 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:52.968098 2577 scope.go:117] "RemoveContainer" containerID="fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be" Jul 10 21:14:52.969174 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:52.969145 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:14:52.969238 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:52.969217 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:14:52.969275 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:52.969250 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:14:53.892988 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:53.892951 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:53.974055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:53.974018 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" event={"ID":"b5f61c5b-9c81-4104-8689-442699160eb1","Type":"ContainerStarted","Data":"06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e"} Jul 10 21:14:54.890371 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:54.890344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:54.891506 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:54.891482 2577 patch_prober.go:28] interesting pod/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 10 21:14:54.891615 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:54.891529 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 10 21:14:55.890302 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:55.890269 2577 patch_prober.go:28] interesting pod/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 10 21:14:55.890700 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:55.890322 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 10 21:14:56.890785 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:56.890746 2577 patch_prober.go:28] interesting pod/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 10 21:14:56.891184 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:56.890805 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 10 21:14:57.890886 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:57.890849 2577 patch_prober.go:28] interesting pod/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 10 21:14:57.891264 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:57.890922 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 10 21:14:58.894207 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:58.894176 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:58.894574 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:58.894416 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:14:58.895039 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:14:58.895020 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:15:00.141562 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:00.141526 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76"] Jul 10 21:15:00.145427 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:00.145405 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" Jul 10 21:15:00.148048 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:00.148026 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:15:00.153073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:00.152811 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76"] Jul 10 21:15:00.156612 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:00.156595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" Jul 10 21:15:00.286058 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:00.286036 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76"] Jul 10 21:15:00.288816 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:15:00.288781 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6568b8d6_1be6_452b_b707_792a0f636e1b.slice/crio-e1f58bd40819a0bd8c47073e419182fc1f214117ed45bd6bba52e9ee935ac88a WatchSource:0}: Error finding container e1f58bd40819a0bd8c47073e419182fc1f214117ed45bd6bba52e9ee935ac88a: Status 404 returned error can't find the container with id e1f58bd40819a0bd8c47073e419182fc1f214117ed45bd6bba52e9ee935ac88a Jul 10 21:15:01.014633 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:01.014600 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" event={"ID":"6568b8d6-1be6-452b-b707-792a0f636e1b","Type":"ContainerStarted","Data":"8748d88537a3b58f37b5bb16d57cc2d01572961efb4ecac5c03b498f38d5baad"} Jul 10 21:15:01.014837 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:01.014686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" event={"ID":"6568b8d6-1be6-452b-b707-792a0f636e1b","Type":"ContainerStarted","Data":"e1f58bd40819a0bd8c47073e419182fc1f214117ed45bd6bba52e9ee935ac88a"} Jul 10 21:15:01.035705 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:01.035635 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" podStartSLOduration=1.035619406 podStartE2EDuration="1.035619406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:15:01.03455612 +0000 UTC m=+2236.382865451" watchObservedRunningTime="2026-07-10 21:15:01.035619406 +0000 UTC m=+2236.383928736" Jul 10 21:15:01.704036 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:01.704003 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76"] Jul 10 21:15:01.710615 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:01.710590 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:02.021084 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:02.021050 2577 generic.go:356] "Generic (PLEG): container finished" podID="6568b8d6-1be6-452b-b707-792a0f636e1b" containerID="8748d88537a3b58f37b5bb16d57cc2d01572961efb4ecac5c03b498f38d5baad" exitCode=0 Jul 10 21:15:02.021248 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:02.021138 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" event={"ID":"6568b8d6-1be6-452b-b707-792a0f636e1b","Type":"ContainerDied","Data":"8748d88537a3b58f37b5bb16d57cc2d01572961efb4ecac5c03b498f38d5baad"} Jul 10 21:15:03.159270 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:03.159241 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" Jul 10 21:15:04.031547 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:04.031517 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" Jul 10 21:15:04.031547 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:04.031540 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76" event={"ID":"6568b8d6-1be6-452b-b707-792a0f636e1b","Type":"ContainerDied","Data":"e1f58bd40819a0bd8c47073e419182fc1f214117ed45bd6bba52e9ee935ac88a"} Jul 10 21:15:04.031847 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:04.031592 2577 scope.go:117] "RemoveContainer" containerID="8748d88537a3b58f37b5bb16d57cc2d01572961efb4ecac5c03b498f38d5baad" Jul 10 21:15:04.051979 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:04.051959 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76"] Jul 10 21:15:04.054151 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:04.054130 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-dd381648-29728635-6xt76"] Jul 10 21:15:04.876826 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:04.876798 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:15:05.164577 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:05.164493 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6568b8d6-1be6-452b-b707-792a0f636e1b" path="/var/lib/kubelet/pods/6568b8d6-1be6-452b-b707-792a0f636e1b/volumes" Jul 10 21:15:06.124168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:06.124139 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:11.807280 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:11.807200 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:11.875622 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:11.875588 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:16.802937 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:16.802902 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f"] Jul 10 21:15:16.803420 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:16.803118 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" containerID="cri-o://06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e" gracePeriod=30 Jul 10 21:15:16.817797 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:16.817772 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:17.057814 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:17.057726 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:17.192793 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:17.192758 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:17.286692 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:17.286645 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:17.413721 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:17.413631 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:22.063377 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.063316 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:15:22.119238 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.119199 2577 generic.go:356] "Generic (PLEG): container finished" podID="b5f61c5b-9c81-4104-8689-442699160eb1" containerID="06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e" exitCode=0 Jul 10 21:15:22.119395 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.119249 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" Jul 10 21:15:22.119395 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.119260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" event={"ID":"b5f61c5b-9c81-4104-8689-442699160eb1","Type":"ContainerDied","Data":"06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e"} Jul 10 21:15:22.119395 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.119330 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f" event={"ID":"b5f61c5b-9c81-4104-8689-442699160eb1","Type":"ContainerDied","Data":"5f4d6555b075b3c0f5f5cca60730bb3c135c308fc84305ca0c33273059b1a11d"} Jul 10 21:15:22.119395 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.119354 2577 scope.go:117] "RemoveContainer" containerID="06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e" Jul 10 21:15:22.128068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.128054 2577 scope.go:117] "RemoveContainer" containerID="fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be" Jul 10 21:15:22.135781 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.135763 2577 scope.go:117] "RemoveContainer" containerID="06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e" Jul 10 21:15:22.136038 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:15:22.136019 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e\": container with ID starting with 06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e not found: ID does not exist" containerID="06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e" Jul 10 21:15:22.136091 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.136047 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e"} err="failed to get container status \"06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e\": rpc error: code = NotFound desc = could not find container \"06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e\": container with ID starting with 06eda853238c556ffb06b85c7ab9739d49fb56969686ce4004ec921a31b9a24e not found: ID does not exist" Jul 10 21:15:22.136091 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.136066 2577 scope.go:117] "RemoveContainer" containerID="fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be" Jul 10 21:15:22.136322 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:15:22.136303 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be\": container with ID starting with fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be not found: ID does not exist" containerID="fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be" Jul 10 21:15:22.136370 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.136329 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be"} err="failed to get container status \"fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be\": rpc error: code = NotFound desc = could not find container \"fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be\": container with ID starting with fe482d1662c1bf31eb280e7fac46a49ef5d2366066843e4d5b09ede1378291be not found: ID does not exist" Jul 10 21:15:22.218388 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-data\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218388 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218397 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5f61c5b-9c81-4104-8689-442699160eb1-istiod-ca-cert\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218611 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218416 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-istio-token\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218611 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218596 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6rpx\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-kube-api-access-j6rpx\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218739 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218700 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-socket\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218738 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-credential-socket\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218779 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5f61c5b-9c81-4104-8689-442699160eb1-istio-podinfo\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218917 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218801 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-certs\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218917 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218827 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-envoy\") pod \"b5f61c5b-9c81-4104-8689-442699160eb1\" (UID: \"b5f61c5b-9c81-4104-8689-442699160eb1\") " Jul 10 21:15:22.218917 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218857 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f61c5b-9c81-4104-8689-442699160eb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:15:22.219086 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.218987 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:15:22.219243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.219194 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.219243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.219202 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:15:22.219243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.219218 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5f61c5b-9c81-4104-8689-442699160eb1-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.219467 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.219259 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:15:22.219467 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.219376 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:15:22.221019 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.220989 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:15:22.221116 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.221038 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:15:22.221240 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.221213 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-kube-api-access-j6rpx" (OuterVolumeSpecName: "kube-api-access-j6rpx") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "kube-api-access-j6rpx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:15:22.221295 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.221243 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5f61c5b-9c81-4104-8689-442699160eb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5f61c5b-9c81-4104-8689-442699160eb1" (UID: "b5f61c5b-9c81-4104-8689-442699160eb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:15:22.320596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320510 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.320596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320541 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.320596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320553 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.320596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320565 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6rpx\" (UniqueName: \"kubernetes.io/projected/b5f61c5b-9c81-4104-8689-442699160eb1-kube-api-access-j6rpx\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.320596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320578 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.320596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320593 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5f61c5b-9c81-4104-8689-442699160eb1-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.320945 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.320606 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5f61c5b-9c81-4104-8689-442699160eb1-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:15:22.459088 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.459047 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f"] Jul 10 21:15:22.479574 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:22.479543 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dd381648-openshift-default-84bfb558c-9mb8f"] Jul 10 21:15:23.164361 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:23.164327 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" path="/var/lib/kubelet/pods/b5f61c5b-9c81-4104-8689-442699160eb1/volumes" Jul 10 21:15:23.451263 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:23.451170 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:23.564510 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:23.564477 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:23.652873 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:23.652843 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:23.782164 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:23.782137 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:30.160893 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:30.160859 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:30.242323 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:30.242289 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:37.025897 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.025855 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:37.345946 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.345857 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26"] Jul 10 21:15:37.346349 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346333 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6568b8d6-1be6-452b-b707-792a0f636e1b" containerName="cleanup" Jul 10 21:15:37.346416 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346353 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6568b8d6-1be6-452b-b707-792a0f636e1b" containerName="cleanup" Jul 10 21:15:37.346416 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346389 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" Jul 10 21:15:37.346416 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346398 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" Jul 10 21:15:37.346416 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346409 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" Jul 10 21:15:37.346610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346418 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" Jul 10 21:15:37.346610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346498 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6568b8d6-1be6-452b-b707-792a0f636e1b" containerName="cleanup" Jul 10 21:15:37.346610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346516 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" Jul 10 21:15:37.346610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.346527 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f61c5b-9c81-4104-8689-442699160eb1" containerName="istio-proxy" Jul 10 21:15:37.351439 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.351416 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.354223 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.354203 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-46c4a127-openshift-default-dockercfg-8dh8s\"" Jul 10 21:15:37.364169 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.364148 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26"] Jul 10 21:15:37.414258 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.414237 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:37.457700 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.457646 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-envoy\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.457900 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.457735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-data\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.457900 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.457832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llrct\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-kube-api-access-llrct\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.457900 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.457891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-certs\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.458084 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.457931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cdf7911-696c-43bf-b373-d27bcc351289-istio-podinfo\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.458084 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.458014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf7911-696c-43bf-b373-d27bcc351289-istiod-ca-cert\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.458084 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.458043 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-istio-token\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.458240 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.458087 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-credential-socket\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.458240 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.458144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-socket\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.558843 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.558804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf7911-696c-43bf-b373-d27bcc351289-istiod-ca-cert\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.558843 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.558847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-istio-token\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.558975 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-credential-socket\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-socket\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-envoy\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-data\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559145 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llrct\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-kube-api-access-llrct\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-certs\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559211 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cdf7911-696c-43bf-b373-d27bcc351289-istio-podinfo\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559496 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559470 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-socket\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559613 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-certs\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559613 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559602 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf7911-696c-43bf-b373-d27bcc351289-istiod-ca-cert\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559756 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-data\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.559756 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.559619 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-credential-socket\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.561367 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.561338 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-envoy\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.561787 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.561766 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cdf7911-696c-43bf-b373-d27bcc351289-istio-podinfo\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.567048 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.567021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-istio-token\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.567231 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.567210 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llrct\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-kube-api-access-llrct\") pod \"e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.663042 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.662950 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:37.807835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.807807 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26"] Jul 10 21:15:37.809762 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:15:37.809732 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdf7911_696c_43bf_b373_d27bcc351289.slice/crio-d7a6041c85267b1da3d62671ad0915cc3ad5571b2ec44bd67b4f08aca54c7be9 WatchSource:0}: Error finding container d7a6041c85267b1da3d62671ad0915cc3ad5571b2ec44bd67b4f08aca54c7be9: Status 404 returned error can't find the container with id d7a6041c85267b1da3d62671ad0915cc3ad5571b2ec44bd67b4f08aca54c7be9 Jul 10 21:15:37.812108 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.812069 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:15:37.812211 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.812153 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:15:37.812249 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:37.812208 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:15:38.005623 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:38.005589 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:38.204036 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:38.203985 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" event={"ID":"6cdf7911-696c-43bf-b373-d27bcc351289","Type":"ContainerStarted","Data":"d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665"} Jul 10 21:15:38.204469 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:38.204078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" event={"ID":"6cdf7911-696c-43bf-b373-d27bcc351289","Type":"ContainerStarted","Data":"d7a6041c85267b1da3d62671ad0915cc3ad5571b2ec44bd67b4f08aca54c7be9"} Jul 10 21:15:38.225967 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:38.225912 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podStartSLOduration=1.225893212 podStartE2EDuration="1.225893212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:15:38.224930534 +0000 UTC m=+2273.573239867" watchObservedRunningTime="2026-07-10 21:15:38.225893212 +0000 UTC m=+2273.574202543" Jul 10 21:15:38.663289 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:38.663248 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:38.668271 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:38.668245 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:39.209443 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:39.209408 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:39.210558 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:39.210534 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:40.211710 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:40.211674 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:40.306100 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:40.306069 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:40.403429 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:40.403395 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:40.880857 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:40.880825 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:40.982115 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:40.982080 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:44.693906 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:44.693866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:45.283480 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:45.283448 2577 scope.go:117] "RemoveContainer" containerID="bda5b04a41d262a3ab5b6ac47a8977b30b3be919465a5a1f1c86927a84835e25" Jul 10 21:15:47.357359 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:47.357310 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:15:49.661882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:49.661842 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:49.824002 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:49.823970 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:49.930644 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:49.930515 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:55.250794 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:55.250754 2577 patch_prober.go:28] interesting pod/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 21:15:55.251334 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:55.250802 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 21:15:55.943228 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:55.943197 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:15:56.291553 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:56.291526 2577 generic.go:356] "Generic (PLEG): container finished" podID="6cdf7911-696c-43bf-b373-d27bcc351289" containerID="d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665" exitCode=0 Jul 10 21:15:56.291917 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:56.291583 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" event={"ID":"6cdf7911-696c-43bf-b373-d27bcc351289","Type":"ContainerDied","Data":"d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665"} Jul 10 21:15:56.291968 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:56.291953 2577 scope.go:117] "RemoveContainer" containerID="d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665" Jul 10 21:15:56.293003 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:56.292978 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:15:56.293107 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:56.293080 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:15:56.293145 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:56.293117 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:15:57.298185 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:57.298145 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" event={"ID":"6cdf7911-696c-43bf-b373-d27bcc351289","Type":"ContainerStarted","Data":"36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45"} Jul 10 21:15:57.663525 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:57.663432 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:15:57.663763 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:57.663730 2577 patch_prober.go:28] interesting pod/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 10 21:15:57.663821 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:57.663804 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 10 21:15:58.664108 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:58.664068 2577 patch_prober.go:28] interesting pod/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 10 21:15:58.664577 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:58.664144 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 10 21:15:59.663977 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:59.663939 2577 patch_prober.go:28] interesting pod/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 10 21:15:59.664134 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:15:59.664028 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 10 21:16:00.663464 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:00.663429 2577 patch_prober.go:28] interesting pod/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 10 21:16:00.663628 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:00.663482 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 10 21:16:01.667195 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:01.667167 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:16:01.667651 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:01.667410 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:16:01.668141 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:01.668122 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:16:02.996729 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:02.996692 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:03.096028 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:03.095997 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:05.152699 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:05.152649 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:16:07.612623 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.612588 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm"] Jul 10 21:16:07.616101 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.616082 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:07.618446 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.618424 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:16:07.618552 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.618458 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:16:07.623420 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.623316 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm"] Jul 10 21:16:07.735022 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.734983 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-46c4a127-rkglm\" (UID: \"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:07.836344 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.836310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-46c4a127-rkglm\" (UID: \"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:07.836952 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.836933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-46c4a127-rkglm\" (UID: \"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:07.927864 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:07.927787 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:08.053423 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:08.053306 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm"] Jul 10 21:16:08.056175 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:16:08.056148 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5b2f9f_2f39_4cb7_9b75_7bbfba21e32e.slice/crio-7557ce72b44cba09f55ed2c4edd89d6e8ab515285e8a75392460dacf6e978360 WatchSource:0}: Error finding container 7557ce72b44cba09f55ed2c4edd89d6e8ab515285e8a75392460dacf6e978360: Status 404 returned error can't find the container with id 7557ce72b44cba09f55ed2c4edd89d6e8ab515285e8a75392460dacf6e978360 Jul 10 21:16:08.351473 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:08.351435 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" event={"ID":"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e","Type":"ContainerStarted","Data":"e17e924ae9f3b3629b6ff08cc8e68ccbc5feabc437e4e6d7dbad1e7f9febf6a5"} Jul 10 21:16:08.351634 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:08.351492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" event={"ID":"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e","Type":"ContainerStarted","Data":"7557ce72b44cba09f55ed2c4edd89d6e8ab515285e8a75392460dacf6e978360"} Jul 10 21:16:08.369557 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:08.369497 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" podStartSLOduration=1.369482453 podStartE2EDuration="1.369482453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:16:08.368479745 +0000 UTC m=+2303.716789086" watchObservedRunningTime="2026-07-10 21:16:08.369482453 +0000 UTC m=+2303.717791788" Jul 10 21:16:09.357243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:09.357212 2577 generic.go:356] "Generic (PLEG): container finished" podID="5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e" containerID="e17e924ae9f3b3629b6ff08cc8e68ccbc5feabc437e4e6d7dbad1e7f9febf6a5" exitCode=0 Jul 10 21:16:09.357633 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:09.357293 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" event={"ID":"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e","Type":"ContainerDied","Data":"e17e924ae9f3b3629b6ff08cc8e68ccbc5feabc437e4e6d7dbad1e7f9febf6a5"} Jul 10 21:16:10.492796 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:10.492774 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:10.560379 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:10.560349 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e-maas-api-service-ca\") pod \"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e\" (UID: \"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e\") " Jul 10 21:16:10.560735 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:10.560706 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e" (UID: "5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:16:10.661399 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:10.661309 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:16:11.367114 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:11.367076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" event={"ID":"5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e","Type":"ContainerDied","Data":"7557ce72b44cba09f55ed2c4edd89d6e8ab515285e8a75392460dacf6e978360"} Jul 10 21:16:11.367114 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:11.367113 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-46c4a127-rkglm" Jul 10 21:16:11.367315 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:11.367135 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7557ce72b44cba09f55ed2c4edd89d6e8ab515285e8a75392460dacf6e978360" Jul 10 21:16:18.198398 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:18.198362 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:22.146146 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:22.146106 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:16:22.734843 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:22.734802 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:27.807497 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:27.807464 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:45.179230 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:16:45.179178 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:04.150110 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:04.150077 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:04.768218 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:04.768184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:10.429327 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:10.429290 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:10.530287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:10.530258 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:14.977307 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:14.977274 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26"] Jul 10 21:17:14.977732 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:14.977502 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" containerID="cri-o://36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45" gracePeriod=30 Jul 10 21:17:15.001468 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:15.001444 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:15.219183 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:15.219145 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:16.381779 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:16.381739 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:16.479849 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:16.479816 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:16.669770 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:16.669680 2577 patch_prober.go:28] interesting pod/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:17:16.669770 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:16.669754 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:17:17.313424 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:17.313393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:17.408896 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:17.408863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:17.506854 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:17.506799 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:18.856998 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:18.856951 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:19.255237 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:19.255202 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:17:20.238727 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.238641 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:17:20.367078 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367043 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-istio-token\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367078 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367084 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf7911-696c-43bf-b373-d27bcc351289-istiod-ca-cert\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367105 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-certs\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367125 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-socket\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367146 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cdf7911-696c-43bf-b373-d27bcc351289-istio-podinfo\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367206 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-data\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367224 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llrct\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-kube-api-access-llrct\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367244 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-credential-socket\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367322 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367298 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-envoy\") pod \"6cdf7911-696c-43bf-b373-d27bcc351289\" (UID: \"6cdf7911-696c-43bf-b373-d27bcc351289\") " Jul 10 21:17:20.367695 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367404 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:20.367695 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367422 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:20.367695 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367567 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdf7911-696c-43bf-b373-d27bcc351289-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:17:20.367695 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367606 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.367695 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367627 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.367974 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.367708 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:20.368188 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.368160 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-data" (OuterVolumeSpecName: "istio-data") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:20.369549 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.369522 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:20.369679 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.369555 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-istio-token" (OuterVolumeSpecName: "istio-token") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:17:20.369920 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.369897 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cdf7911-696c-43bf-b373-d27bcc351289-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:17:20.369920 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.369909 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-kube-api-access-llrct" (OuterVolumeSpecName: "kube-api-access-llrct") pod "6cdf7911-696c-43bf-b373-d27bcc351289" (UID: "6cdf7911-696c-43bf-b373-d27bcc351289"). InnerVolumeSpecName "kube-api-access-llrct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:17:20.468485 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468453 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.468485 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468478 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf7911-696c-43bf-b373-d27bcc351289-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.468485 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468489 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cdf7911-696c-43bf-b373-d27bcc351289-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.468725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468497 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.468725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468506 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llrct\" (UniqueName: \"kubernetes.io/projected/6cdf7911-696c-43bf-b373-d27bcc351289-kube-api-access-llrct\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.468725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468516 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.468725 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.468525 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cdf7911-696c-43bf-b373-d27bcc351289-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:17:20.706427 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.706396 2577 generic.go:356] "Generic (PLEG): container finished" podID="6cdf7911-696c-43bf-b373-d27bcc351289" containerID="36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45" exitCode=0 Jul 10 21:17:20.706598 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.706445 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" event={"ID":"6cdf7911-696c-43bf-b373-d27bcc351289","Type":"ContainerDied","Data":"36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45"} Jul 10 21:17:20.706598 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.706485 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" event={"ID":"6cdf7911-696c-43bf-b373-d27bcc351289","Type":"ContainerDied","Data":"d7a6041c85267b1da3d62671ad0915cc3ad5571b2ec44bd67b4f08aca54c7be9"} Jul 10 21:17:20.706598 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.706504 2577 scope.go:117] "RemoveContainer" containerID="36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45" Jul 10 21:17:20.706598 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.706449 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26" Jul 10 21:17:20.716031 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.716013 2577 scope.go:117] "RemoveContainer" containerID="d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665" Jul 10 21:17:20.723746 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.723729 2577 scope.go:117] "RemoveContainer" containerID="36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45" Jul 10 21:17:20.723985 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:17:20.723965 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45\": container with ID starting with 36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45 not found: ID does not exist" containerID="36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45" Jul 10 21:17:20.724036 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.723993 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45"} err="failed to get container status \"36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45\": rpc error: code = NotFound desc = could not find container \"36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45\": container with ID starting with 36b4f6681590c22621ee9bbeb090d373f3de529b92241ddd90e13818eebf4d45 not found: ID does not exist" Jul 10 21:17:20.724036 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.724008 2577 scope.go:117] "RemoveContainer" containerID="d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665" Jul 10 21:17:20.724225 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:17:20.724204 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665\": container with ID starting with d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665 not found: ID does not exist" containerID="d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665" Jul 10 21:17:20.724280 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.724230 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665"} err="failed to get container status \"d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665\": rpc error: code = NotFound desc = could not find container \"d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665\": container with ID starting with d716554de36ba116c5062813db205d53a86775381e9c22ae1f49f113de9f2665 not found: ID does not exist" Jul 10 21:17:20.732204 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.732179 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26"] Jul 10 21:17:20.738860 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:20.738838 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-46c4a127-openshift-default-5df7d597ff-rbc26"] Jul 10 21:17:21.163767 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:21.163735 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" path="/var/lib/kubelet/pods/6cdf7911-696c-43bf-b373-d27bcc351289/volumes" Jul 10 21:17:24.460987 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:24.460953 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:25.971291 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:25.971247 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:30.862117 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:30.862075 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:17:30.976856 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:30.976825 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:31.036491 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:31.036452 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:31.115526 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:31.115427 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:31.214611 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:31.214584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:41.721992 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:41.721949 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:42.055536 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:17:42.055499 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:03.067013 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:03.066980 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:08.811916 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:08.811837 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:08.912016 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:08.911984 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:09.257032 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.257004 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:09.539510 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.539429 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m"] Jul 10 21:18:09.539897 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.539871 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" Jul 10 21:18:09.539897 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.539897 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" Jul 10 21:18:09.540082 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.539929 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e" containerName="revoke-keys" Jul 10 21:18:09.540082 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.539936 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e" containerName="revoke-keys" Jul 10 21:18:09.540082 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.540005 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" Jul 10 21:18:09.540082 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.540017 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e" containerName="revoke-keys" Jul 10 21:18:09.540082 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.540023 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" Jul 10 21:18:09.543027 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.543007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.545539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.545517 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-42c194-openshift-default-dockercfg-t7wqx\"" Jul 10 21:18:09.557115 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.557090 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m"] Jul 10 21:18:09.562451 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.562430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:09.598417 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-envoy\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598592 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-data\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598592 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-token\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598592 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598523 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7p65\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-kube-api-access-n7p65\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598768 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-podinfo\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598768 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0fc512-244d-4a36-92e4-111ea1baa762-istiod-ca-cert\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598768 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-credential-socket\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598893 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598784 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-socket\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.598893 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.598801 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-certs\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.700299 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.700255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-socket\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.700551 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.700534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-certs\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.700716 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.700702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-envoy\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.700854 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.700826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-data\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.700986 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.700969 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-token\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.701087 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.701074 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7p65\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-kube-api-access-n7p65\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.701239 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.701225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-podinfo\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.701338 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.701324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0fc512-244d-4a36-92e4-111ea1baa762-istiod-ca-cert\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.701442 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.701430 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-credential-socket\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.701977 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.701957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-credential-socket\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.702353 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.702335 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-socket\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.702697 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.702676 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-certs\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.704320 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.704295 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-data\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.707943 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.707917 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0fc512-244d-4a36-92e4-111ea1baa762-istiod-ca-cert\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.708339 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.708319 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-envoy\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.709205 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.709180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-podinfo\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.712699 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.712647 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-token\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.712780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.712742 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7p65\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-kube-api-access-n7p65\") pod \"e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.828495 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.828412 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:09.853455 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.853426 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:09.995518 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.995489 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m"] Jul 10 21:18:09.997678 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:18:09.997638 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0fc512_244d_4a36_92e4_111ea1baa762.slice/crio-8b064bc9c3f2521e0850fd8a8e2a78985829c1f266756675188c7e235397fdee WatchSource:0}: Error finding container 8b064bc9c3f2521e0850fd8a8e2a78985829c1f266756675188c7e235397fdee: Status 404 returned error can't find the container with id 8b064bc9c3f2521e0850fd8a8e2a78985829c1f266756675188c7e235397fdee Jul 10 21:18:09.999856 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.999829 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:09.999928 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.999889 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:09.999928 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:09.999916 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:10.204705 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:10.204594 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:10.305775 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:10.305740 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:10.940523 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:10.940493 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" event={"ID":"fb0fc512-244d-4a36-92e4-111ea1baa762","Type":"ContainerStarted","Data":"0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc"} Jul 10 21:18:10.940939 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:10.940542 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" event={"ID":"fb0fc512-244d-4a36-92e4-111ea1baa762","Type":"ContainerStarted","Data":"8b064bc9c3f2521e0850fd8a8e2a78985829c1f266756675188c7e235397fdee"} Jul 10 21:18:10.961907 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:10.961844 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podStartSLOduration=1.961826048 podStartE2EDuration="1.961826048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:18:10.960965808 +0000 UTC m=+2426.309275135" watchObservedRunningTime="2026-07-10 21:18:10.961826048 +0000 UTC m=+2426.310135376" Jul 10 21:18:11.854103 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:11.854068 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:11.859350 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:11.859327 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:11.945192 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:11.945160 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:11.946018 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:11.945998 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:12.972699 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:12.972643 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:13.068085 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:13.068052 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:13.144846 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:13.144815 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:13.482774 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:13.482741 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:13.782227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:13.782193 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:13.882959 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:13.882925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:17.483521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:17.483486 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:17.840182 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:17.840154 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:18.132377 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.132290 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5"] Jul 10 21:18:18.132671 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.132641 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" Jul 10 21:18:18.132819 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.132680 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdf7911-696c-43bf-b373-d27bcc351289" containerName="istio-proxy" Jul 10 21:18:18.136147 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.136130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.138583 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.138565 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5f5b2e-openshift-default-dockercfg-k7lzt\"" Jul 10 21:18:18.148389 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.148367 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5"] Jul 10 21:18:18.170868 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.170842 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:18.278938 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.278898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-socket\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279135 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-envoy\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279242 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-certs\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279311 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae56460-d99f-43d0-9500-5bfdbeadca8f-istiod-ca-cert\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279369 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7sfd\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-kube-api-access-t7sfd\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279369 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-data\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279453 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-podinfo\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279505 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279486 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-credential-socket\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.279542 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.279523 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-token\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380507 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380477 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-envoy\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380681 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-certs\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380681 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae56460-d99f-43d0-9500-5bfdbeadca8f-istiod-ca-cert\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380681 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7sfd\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-kube-api-access-t7sfd\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380681 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380570 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-data\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-podinfo\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-credential-socket\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-token\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.380910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.380860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-socket\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.381118 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.381012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-certs\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.381118 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.381027 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-data\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.381328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.381306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae56460-d99f-43d0-9500-5bfdbeadca8f-istiod-ca-cert\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.381448 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.381420 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-socket\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.381558 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.381460 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-credential-socket\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.382925 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.382875 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-envoy\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.383300 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.383276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-podinfo\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.388299 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.388274 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-token\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.388387 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.388366 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7sfd\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-kube-api-access-t7sfd\") pod \"e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.447747 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.447722 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:18.458889 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.458858 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:18.600528 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.600495 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5"] Jul 10 21:18:18.601475 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:18:18.601447 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae56460_d99f_43d0_9500_5bfdbeadca8f.slice/crio-e215e191d4ddc7a5d4aa3f22676d5997516bcc7868d3a1886a0de9719be62285 WatchSource:0}: Error finding container e215e191d4ddc7a5d4aa3f22676d5997516bcc7868d3a1886a0de9719be62285: Status 404 returned error can't find the container with id e215e191d4ddc7a5d4aa3f22676d5997516bcc7868d3a1886a0de9719be62285 Jul 10 21:18:18.604087 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.604054 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:18.604191 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.604129 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:18.604191 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.604172 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:18.985560 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.985476 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" event={"ID":"dae56460-d99f-43d0-9500-5bfdbeadca8f","Type":"ContainerStarted","Data":"a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05"} Jul 10 21:18:18.985560 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:18.985527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" event={"ID":"dae56460-d99f-43d0-9500-5bfdbeadca8f","Type":"ContainerStarted","Data":"e215e191d4ddc7a5d4aa3f22676d5997516bcc7868d3a1886a0de9719be62285"} Jul 10 21:18:19.005097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:19.005053 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podStartSLOduration=1.005037792 podStartE2EDuration="1.005037792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:18:19.003470764 +0000 UTC m=+2434.351780095" watchObservedRunningTime="2026-07-10 21:18:19.005037792 +0000 UTC m=+2434.353347194" Jul 10 21:18:19.448060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:19.448026 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:19.452590 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:19.452565 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:19.990346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:19.990318 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:19.991320 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:19.991298 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:20.650308 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:20.650281 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:18:20.933510 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:20.933434 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:21.005714 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:21.005679 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:21.312517 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:21.312488 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:21.533424 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:21.533387 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:22.180355 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:22.180317 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:22.281227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:22.281192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:25.403721 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:25.403688 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:27.986813 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:27.986773 2577 patch_prober.go:28] interesting pod/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 21:18:27.987188 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:27.986837 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 21:18:29.062013 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:29.061922 2577 generic.go:356] "Generic (PLEG): container finished" podID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerID="0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc" exitCode=0 Jul 10 21:18:29.062524 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:29.062006 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" event={"ID":"fb0fc512-244d-4a36-92e4-111ea1baa762","Type":"ContainerDied","Data":"0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc"} Jul 10 21:18:29.062595 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:29.062533 2577 scope.go:117] "RemoveContainer" containerID="0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc" Jul 10 21:18:29.063563 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:29.063535 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:29.063680 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:29.063614 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:29.063755 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:29.063692 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:30.072028 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:30.071984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" event={"ID":"fb0fc512-244d-4a36-92e4-111ea1baa762","Type":"ContainerStarted","Data":"9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020"} Jul 10 21:18:30.854538 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:30.854502 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:30.854757 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:30.854620 2577 patch_prober.go:28] interesting pod/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 10 21:18:30.854757 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:30.854681 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 10 21:18:31.854472 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:31.854423 2577 patch_prober.go:28] interesting pod/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 10 21:18:31.855009 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:31.854496 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 10 21:18:32.147766 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:32.147634 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:18:32.854184 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:32.854143 2577 patch_prober.go:28] interesting pod/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 10 21:18:32.854359 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:32.854213 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 10 21:18:33.854102 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:33.854040 2577 patch_prober.go:28] interesting pod/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 10 21:18:33.854102 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:33.854100 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 10 21:18:34.815150 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:34.815113 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:34.857994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:34.857968 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:34.858346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:34.858237 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:34.859025 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:34.859001 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:18:34.881585 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:34.881554 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:35.210573 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:35.210218 2577 patch_prober.go:28] interesting pod/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:45622->10.133.0.43:15021: read: connection reset by peer" start-of-body= Jul 10 21:18:35.210573 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:35.210277 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:45622->10.133.0.43:15021: read: connection reset by peer" Jul 10 21:18:36.106414 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:36.106372 2577 generic.go:356] "Generic (PLEG): container finished" podID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerID="a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05" exitCode=0 Jul 10 21:18:36.106810 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:36.106445 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" event={"ID":"dae56460-d99f-43d0-9500-5bfdbeadca8f","Type":"ContainerDied","Data":"a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05"} Jul 10 21:18:36.107013 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:36.106997 2577 scope.go:117] "RemoveContainer" containerID="a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05" Jul 10 21:18:36.107966 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:36.107942 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:36.108027 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:36.108004 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:36.108066 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:36.108037 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:18:37.113443 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:37.113411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" event={"ID":"dae56460-d99f-43d0-9500-5bfdbeadca8f","Type":"ContainerStarted","Data":"93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2"} Jul 10 21:18:37.448227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:37.448138 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:37.448575 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:37.448492 2577 patch_prober.go:28] interesting pod/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 10 21:18:37.448767 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:37.448744 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 10 21:18:38.448156 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:38.448111 2577 patch_prober.go:28] interesting pod/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 10 21:18:38.448648 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:38.448187 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 10 21:18:39.449138 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:39.449088 2577 patch_prober.go:28] interesting pod/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 10 21:18:39.449685 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:39.449164 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 10 21:18:40.449032 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:40.448987 2577 patch_prober.go:28] interesting pod/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 10 21:18:40.449243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:40.449058 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 10 21:18:41.451673 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:41.451625 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:41.452107 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:41.451921 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:41.452521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:41.452504 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:18:42.648954 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:42.648924 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:42.750352 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:42.750322 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:49.049595 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:49.049558 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:52.252735 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:52.252703 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:18:55.115925 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:18:55.115889 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:04.055452 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:04.055416 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:19:14.097473 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:14.097443 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:18.651378 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:18.651336 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:19:22.176111 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:22.176073 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:33.051563 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:33.051530 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:19:34.177138 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:34.177109 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:35.163864 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.163820 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:35.275642 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.275605 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv"] Jul 10 21:19:35.279334 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.279317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:35.281839 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.281815 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9j9gg\"" Jul 10 21:19:35.281971 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.281817 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:19:35.286162 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.286133 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv"] Jul 10 21:19:35.404587 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.404556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a7100-5b05-443e-9c98-2f10161ed2f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv\" (UID: \"356a7100-5b05-443e-9c98-2f10161ed2f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:35.505573 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.505544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a7100-5b05-443e-9c98-2f10161ed2f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv\" (UID: \"356a7100-5b05-443e-9c98-2f10161ed2f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:35.506170 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.506151 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a7100-5b05-443e-9c98-2f10161ed2f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv\" (UID: \"356a7100-5b05-443e-9c98-2f10161ed2f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:35.591369 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.591334 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:35.724208 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.724178 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv"] Jul 10 21:19:35.725532 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:19:35.725507 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356a7100_5b05_443e_9c98_2f10161ed2f4.slice/crio-9ed1c7f315645a0ea41af2ae998d0e6688f03f111317ff2d725078ca7cefc7fe WatchSource:0}: Error finding container 9ed1c7f315645a0ea41af2ae998d0e6688f03f111317ff2d725078ca7cefc7fe: Status 404 returned error can't find the container with id 9ed1c7f315645a0ea41af2ae998d0e6688f03f111317ff2d725078ca7cefc7fe Jul 10 21:19:35.898052 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:35.898019 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:36.001118 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:36.001087 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:36.401548 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:36.401516 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" event={"ID":"356a7100-5b05-443e-9c98-2f10161ed2f4","Type":"ContainerStarted","Data":"b84b94473ef24951d8a75fe38c25f811c3df4450a9dc4a77a83040c0cabc6411"} Jul 10 21:19:36.401935 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:36.401570 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" event={"ID":"356a7100-5b05-443e-9c98-2f10161ed2f4","Type":"ContainerStarted","Data":"9ed1c7f315645a0ea41af2ae998d0e6688f03f111317ff2d725078ca7cefc7fe"} Jul 10 21:19:36.417740 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:36.417694 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" podStartSLOduration=1.417649995 podStartE2EDuration="1.417649995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:19:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:19:36.41687405 +0000 UTC m=+2511.765183379" watchObservedRunningTime="2026-07-10 21:19:36.417649995 +0000 UTC m=+2511.765959325" Jul 10 21:19:37.407158 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:37.407070 2577 generic.go:356] "Generic (PLEG): container finished" podID="356a7100-5b05-443e-9c98-2f10161ed2f4" containerID="b84b94473ef24951d8a75fe38c25f811c3df4450a9dc4a77a83040c0cabc6411" exitCode=0 Jul 10 21:19:37.407508 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:37.407159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" event={"ID":"356a7100-5b05-443e-9c98-2f10161ed2f4","Type":"ContainerDied","Data":"b84b94473ef24951d8a75fe38c25f811c3df4450a9dc4a77a83040c0cabc6411"} Jul 10 21:19:38.545315 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:38.545294 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:38.630893 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:38.630861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a7100-5b05-443e-9c98-2f10161ed2f4-maas-api-service-ca\") pod \"356a7100-5b05-443e-9c98-2f10161ed2f4\" (UID: \"356a7100-5b05-443e-9c98-2f10161ed2f4\") " Jul 10 21:19:38.631260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:38.631231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/356a7100-5b05-443e-9c98-2f10161ed2f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "356a7100-5b05-443e-9c98-2f10161ed2f4" (UID: "356a7100-5b05-443e-9c98-2f10161ed2f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:19:38.732136 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:38.732057 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/356a7100-5b05-443e-9c98-2f10161ed2f4-maas-api-service-ca\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:19:39.417613 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:39.417585 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" Jul 10 21:19:39.417785 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:39.417621 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv" event={"ID":"356a7100-5b05-443e-9c98-2f10161ed2f4","Type":"ContainerDied","Data":"9ed1c7f315645a0ea41af2ae998d0e6688f03f111317ff2d725078ca7cefc7fe"} Jul 10 21:19:39.417785 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:39.417684 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ed1c7f315645a0ea41af2ae998d0e6688f03f111317ff2d725078ca7cefc7fe" Jul 10 21:19:45.358019 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:45.357992 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:19:45.780098 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:45.780069 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:50.389446 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:50.389399 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:55.559086 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:55.559053 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:19:56.180994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:56.180937 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:19:56.281235 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:19:56.281201 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:01.124842 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:01.124806 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m"] Jul 10 21:20:01.125221 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:01.125045 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" containerID="cri-o://9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020" gracePeriod=30 Jul 10 21:20:01.188705 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:01.188649 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:01.433469 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:01.433389 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:02.416549 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:02.416509 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:02.518913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:02.518849 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:03.910876 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:03.910845 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:04.861171 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:04.861131 2577 patch_prober.go:28] interesting pod/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:20:04.861360 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:04.861207 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:20:06.389124 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.389065 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:20:06.492966 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.492935 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-certs\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493154 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.492975 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0fc512-244d-4a36-92e4-111ea1baa762-istiod-ca-cert\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493154 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493016 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-token\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493154 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493041 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-podinfo\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493154 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493117 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-credential-socket\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493158 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-envoy\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493189 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7p65\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-kube-api-access-n7p65\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493212 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:06.493373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493238 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-socket\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493373 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493300 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-data\") pod \"fb0fc512-244d-4a36-92e4-111ea1baa762\" (UID: \"fb0fc512-244d-4a36-92e4-111ea1baa762\") " Jul 10 21:20:06.493734 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493419 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:06.493734 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493481 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb0fc512-244d-4a36-92e4-111ea1baa762-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:20:06.493854 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493768 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:06.493909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493851 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.493909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493872 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.493909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493887 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.493909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.493900 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0fc512-244d-4a36-92e4-111ea1baa762-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.494252 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.494231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:06.495333 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.495310 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:20:06.495431 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.495411 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:06.495742 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.495711 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:20:06.495742 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.495726 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-kube-api-access-n7p65" (OuterVolumeSpecName: "kube-api-access-n7p65") pod "fb0fc512-244d-4a36-92e4-111ea1baa762" (UID: "fb0fc512-244d-4a36-92e4-111ea1baa762"). InnerVolumeSpecName "kube-api-access-n7p65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:20:06.553709 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.553680 2577 generic.go:356] "Generic (PLEG): container finished" podID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerID="9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020" exitCode=0 Jul 10 21:20:06.553840 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.553784 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" Jul 10 21:20:06.553840 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.553809 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" event={"ID":"fb0fc512-244d-4a36-92e4-111ea1baa762","Type":"ContainerDied","Data":"9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020"} Jul 10 21:20:06.553916 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.553853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m" event={"ID":"fb0fc512-244d-4a36-92e4-111ea1baa762","Type":"ContainerDied","Data":"8b064bc9c3f2521e0850fd8a8e2a78985829c1f266756675188c7e235397fdee"} Jul 10 21:20:06.553916 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.553877 2577 scope.go:117] "RemoveContainer" containerID="9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020" Jul 10 21:20:06.562892 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.562876 2577 scope.go:117] "RemoveContainer" containerID="0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc" Jul 10 21:20:06.570328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.570311 2577 scope.go:117] "RemoveContainer" containerID="9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020" Jul 10 21:20:06.570548 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:20:06.570529 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020\": container with ID starting with 9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020 not found: ID does not exist" containerID="9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020" Jul 10 21:20:06.570594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.570557 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020"} err="failed to get container status \"9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020\": rpc error: code = NotFound desc = could not find container \"9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020\": container with ID starting with 9cedce7d2ab21f021e0429fd227906a5384773bfcd054185395a4cfd6a6cc020 not found: ID does not exist" Jul 10 21:20:06.570594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.570574 2577 scope.go:117] "RemoveContainer" containerID="0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc" Jul 10 21:20:06.570813 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:20:06.570791 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc\": container with ID starting with 0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc not found: ID does not exist" containerID="0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc" Jul 10 21:20:06.570861 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.570820 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc"} err="failed to get container status \"0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc\": rpc error: code = NotFound desc = could not find container \"0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc\": container with ID starting with 0e14f65eb1e52d6990aa30c8847ce5ae016e87de8dbac45a4f9844a8f96000fc not found: ID does not exist" Jul 10 21:20:06.577105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.577083 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m"] Jul 10 21:20:06.581176 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.581150 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-42c194-openshift-default-79b884cfff-jvr8m"] Jul 10 21:20:06.595130 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.595109 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.595130 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.595129 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.595242 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.595138 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.595242 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.595146 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0fc512-244d-4a36-92e4-111ea1baa762-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:06.595242 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:06.595155 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7p65\" (UniqueName: \"kubernetes.io/projected/fb0fc512-244d-4a36-92e4-111ea1baa762-kube-api-access-n7p65\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:07.165125 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:07.165092 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" path="/var/lib/kubelet/pods/fb0fc512-244d-4a36-92e4-111ea1baa762/volumes" Jul 10 21:20:12.069080 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:12.069041 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:15.234986 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:15.234946 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:20:16.615107 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:16.615076 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:22.641885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:22.641853 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:22.701767 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:22.701734 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:27.311798 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:27.311762 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5"] Jul 10 21:20:27.312353 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:27.311969 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" containerID="cri-o://93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2" gracePeriod=30 Jul 10 21:20:27.338361 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:27.338335 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:27.560519 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:27.560479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:27.903276 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:27.903241 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:28.186215 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186129 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2"] Jul 10 21:20:28.186500 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186487 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" Jul 10 21:20:28.186564 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186501 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" Jul 10 21:20:28.186564 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186523 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="356a7100-5b05-443e-9c98-2f10161ed2f4" containerName="revoke-keys" Jul 10 21:20:28.186564 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186529 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="356a7100-5b05-443e-9c98-2f10161ed2f4" containerName="revoke-keys" Jul 10 21:20:28.186754 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186598 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" Jul 10 21:20:28.186754 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.186611 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="356a7100-5b05-443e-9c98-2f10161ed2f4" containerName="revoke-keys" Jul 10 21:20:28.189886 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.189861 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.192520 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.192496 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ed1347-openshift-default-dockercfg-h5gqx\"" Jul 10 21:20:28.202901 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.202876 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2"] Jul 10 21:20:28.232019 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.231990 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:28.297673 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297629 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-podinfo\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.297673 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-data\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.297866 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-certs\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.297866 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-envoy\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.297866 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-token\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.297968 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-credential-socket\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.297968 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.297909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfh5n\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-kube-api-access-bfh5n\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.298084 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.298066 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istiod-ca-cert\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.298125 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.298095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-socket\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.307835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.307811 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:28.399871 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.399827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-data\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.399897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-certs\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.399933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-envoy\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.399991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-token\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.400024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-credential-socket\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.400055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfh5n\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-kube-api-access-bfh5n\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.400149 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istiod-ca-cert\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.400185 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-socket\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.400346 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.400243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-podinfo\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.401064 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.401040 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-certs\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.401165 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.401112 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-data\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.402028 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.402004 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-credential-socket\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.403035 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.402493 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-socket\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.403035 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.402955 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istiod-ca-cert\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.407913 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.407885 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-envoy\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.408400 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.408378 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-podinfo\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.412863 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.412838 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-token\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.413269 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.413249 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfh5n\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-kube-api-access-bfh5n\") pod \"e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.501052 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.501017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:28.635396 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.635368 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2"] Jul 10 21:20:28.637069 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:20:28.637041 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad01fc8_62eb_4df2_b905_f1f3c1fb667f.slice/crio-c18e23864ffe3e947b54102ce4d4c93084c371baff28368c3ee787473db7bf6c WatchSource:0}: Error finding container c18e23864ffe3e947b54102ce4d4c93084c371baff28368c3ee787473db7bf6c: Status 404 returned error can't find the container with id c18e23864ffe3e947b54102ce4d4c93084c371baff28368c3ee787473db7bf6c Jul 10 21:20:28.639311 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.639277 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:28.639414 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.639347 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:28.639414 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.639377 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:28.662068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:28.662043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" event={"ID":"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f","Type":"ContainerStarted","Data":"c18e23864ffe3e947b54102ce4d4c93084c371baff28368c3ee787473db7bf6c"} Jul 10 21:20:29.667851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:29.667819 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" event={"ID":"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f","Type":"ContainerStarted","Data":"34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727"} Jul 10 21:20:29.689757 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:29.689711 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podStartSLOduration=1.689695513 podStartE2EDuration="1.689695513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:20:29.687950028 +0000 UTC m=+2565.036259359" watchObservedRunningTime="2026-07-10 21:20:29.689695513 +0000 UTC m=+2565.038004842" Jul 10 21:20:30.502044 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:30.502002 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:30.506885 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:30.506860 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:30.672731 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:30.672697 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:30.673587 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:30.673568 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:31.649238 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:31.649209 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:31.751924 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:31.751893 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:31.871594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:31.871563 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:31.997200 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:31.997170 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:32.570866 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.570807 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:20:32.664818 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.664785 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:32.683385 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.683358 2577 generic.go:356] "Generic (PLEG): container finished" podID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerID="93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2" exitCode=0 Jul 10 21:20:32.683567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.683413 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" Jul 10 21:20:32.683567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.683433 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" event={"ID":"dae56460-d99f-43d0-9500-5bfdbeadca8f","Type":"ContainerDied","Data":"93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2"} Jul 10 21:20:32.683567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.683492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5" event={"ID":"dae56460-d99f-43d0-9500-5bfdbeadca8f","Type":"ContainerDied","Data":"e215e191d4ddc7a5d4aa3f22676d5997516bcc7868d3a1886a0de9719be62285"} Jul 10 21:20:32.683567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.683517 2577 scope.go:117] "RemoveContainer" containerID="93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2" Jul 10 21:20:32.694341 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.694318 2577 scope.go:117] "RemoveContainer" containerID="a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05" Jul 10 21:20:32.702381 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.702366 2577 scope.go:117] "RemoveContainer" containerID="93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2" Jul 10 21:20:32.702633 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:20:32.702616 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2\": container with ID starting with 93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2 not found: ID does not exist" containerID="93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2" Jul 10 21:20:32.702767 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.702643 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2"} err="failed to get container status \"93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2\": rpc error: code = NotFound desc = could not find container \"93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2\": container with ID starting with 93c7566f3d86a8f00610e72e4d81d6a6c97da72c0cbd1f7db7ba9318873367e2 not found: ID does not exist" Jul 10 21:20:32.702767 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.702679 2577 scope.go:117] "RemoveContainer" containerID="a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05" Jul 10 21:20:32.702907 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:20:32.702890 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05\": container with ID starting with a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05 not found: ID does not exist" containerID="a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05" Jul 10 21:20:32.702944 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.702915 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05"} err="failed to get container status \"a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05\": rpc error: code = NotFound desc = could not find container \"a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05\": container with ID starting with a7abf4f8467af6d489076ac98997691ab50bc11d7ec07a150d2d0839468aaf05 not found: ID does not exist" Jul 10 21:20:32.736002 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.735974 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:32.744051 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744032 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-token\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744121 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744063 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-certs\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744121 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744081 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-socket\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744194 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744120 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7sfd\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-kube-api-access-t7sfd\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744194 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744165 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-envoy\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744280 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744211 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-podinfo\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744280 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744257 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-data\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744387 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744282 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-credential-socket\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744387 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744310 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:32.744387 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744342 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:32.744542 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744391 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae56460-d99f-43d0-9500-5bfdbeadca8f-istiod-ca-cert\") pod \"dae56460-d99f-43d0-9500-5bfdbeadca8f\" (UID: \"dae56460-d99f-43d0-9500-5bfdbeadca8f\") " Jul 10 21:20:32.744598 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744538 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:32.744828 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744807 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.744910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744838 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.744910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744851 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.744910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.744808 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dae56460-d99f-43d0-9500-5bfdbeadca8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:20:32.745525 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.745493 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:32.746403 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.746366 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:20:32.746572 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.746550 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:20:32.746644 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.746592 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:20:32.746861 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.746838 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-kube-api-access-t7sfd" (OuterVolumeSpecName: "kube-api-access-t7sfd") pod "dae56460-d99f-43d0-9500-5bfdbeadca8f" (UID: "dae56460-d99f-43d0-9500-5bfdbeadca8f"). InnerVolumeSpecName "kube-api-access-t7sfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:20:32.845956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.845781 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dae56460-d99f-43d0-9500-5bfdbeadca8f-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.845956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.845818 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.845956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.845832 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7sfd\" (UniqueName: \"kubernetes.io/projected/dae56460-d99f-43d0-9500-5bfdbeadca8f-kube-api-access-t7sfd\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.845956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.845847 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.845956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.845860 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:32.845956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:32.845872 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dae56460-d99f-43d0-9500-5bfdbeadca8f-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:20:33.007903 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:33.007874 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5"] Jul 10 21:20:33.015504 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:33.015481 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5f5b2e-openshift-default-df54ffffc-shhh5"] Jul 10 21:20:33.169789 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:33.164555 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" path="/var/lib/kubelet/pods/dae56460-d99f-43d0-9500-5bfdbeadca8f/volumes" Jul 10 21:20:36.145383 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.145349 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:36.639146 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.639111 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:36.914886 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.914794 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn"] Jul 10 21:20:36.915168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.915154 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" Jul 10 21:20:36.915230 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.915170 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" Jul 10 21:20:36.915230 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.915196 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" Jul 10 21:20:36.915230 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.915201 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" Jul 10 21:20:36.915381 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.915263 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb0fc512-244d-4a36-92e4-111ea1baa762" containerName="istio-proxy" Jul 10 21:20:36.915381 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.915274 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" Jul 10 21:20:36.919985 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.919961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.922892 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.922872 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c4ebda-openshift-default-dockercfg-xg88c\"" Jul 10 21:20:36.933928 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.933904 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn"] Jul 10 21:20:36.941908 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.941886 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:36.983925 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.983888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-socket\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.983941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-credential-socket\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.983976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-data\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.984050 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-envoy\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.984076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istiod-ca-cert\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984239 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.984117 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-podinfo\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984239 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.984187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-token\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984239 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.984222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-certs\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:36.984332 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:36.984246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcd6j\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-kube-api-access-kcd6j\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085047 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-credential-socket\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085047 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-data\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085294 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085077 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-envoy\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085294 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istiod-ca-cert\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085294 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-podinfo\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085459 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085307 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-token\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085459 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085340 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-certs\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085459 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085364 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcd6j\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-kube-api-access-kcd6j\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085459 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-socket\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085459 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-data\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085999 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-socket\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.085999 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.085986 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-credential-socket\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.086179 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.086001 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istiod-ca-cert\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.086179 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.086029 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-certs\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.087759 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.087730 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-podinfo\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.087759 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.087736 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-envoy\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.094485 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.094467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-token\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.094594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.094567 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcd6j\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-kube-api-access-kcd6j\") pod \"e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.230717 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.230619 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:37.294920 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.294879 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:37.360769 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.360730 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:20:37.380836 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.380805 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn"] Jul 10 21:20:37.385139 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:20:37.385108 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1de2a1_9499_4a31_9cf3_d450524bc4db.slice/crio-413070a3c329f20a2a9206aac43802f06f5f131f1f44c7ddbb22a5325e386d40 WatchSource:0}: Error finding container 413070a3c329f20a2a9206aac43802f06f5f131f1f44c7ddbb22a5325e386d40: Status 404 returned error can't find the container with id 413070a3c329f20a2a9206aac43802f06f5f131f1f44c7ddbb22a5325e386d40 Jul 10 21:20:37.386949 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.386912 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:37.387037 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.387015 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:37.387073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.387054 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:37.712496 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.712459 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" event={"ID":"6e1de2a1-9499-4a31-9cf3-d450524bc4db","Type":"ContainerStarted","Data":"28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09"} Jul 10 21:20:37.712731 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.712525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" event={"ID":"6e1de2a1-9499-4a31-9cf3-d450524bc4db","Type":"ContainerStarted","Data":"413070a3c329f20a2a9206aac43802f06f5f131f1f44c7ddbb22a5325e386d40"} Jul 10 21:20:37.735407 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:37.735341 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" podStartSLOduration=1.7353196720000001 podStartE2EDuration="1.735319672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:20:37.734827512 +0000 UTC m=+2573.083136845" watchObservedRunningTime="2026-07-10 21:20:37.735319672 +0000 UTC m=+2573.083629005" Jul 10 21:20:38.233451 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:38.231786 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:38.239048 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:38.238983 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:38.718014 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:38.717984 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:38.718977 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:38.718954 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:39.647309 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:39.647272 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:39.741468 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:39.741427 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:39.824694 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:39.824650 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:40.387106 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:40.387075 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:40.498331 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:40.498300 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:40.598512 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:40.598476 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:44.183621 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:44.183577 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:45.674209 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.674120 2577 patch_prober.go:28] interesting pod/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 10 21:20:45.674209 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.674177 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 10 21:20:45.762024 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.761858 2577 generic.go:356] "Generic (PLEG): container finished" podID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerID="34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727" exitCode=0 Jul 10 21:20:45.762024 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.761993 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" event={"ID":"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f","Type":"ContainerDied","Data":"34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727"} Jul 10 21:20:45.762495 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.762477 2577 scope.go:117] "RemoveContainer" containerID="34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727" Jul 10 21:20:45.763712 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.763676 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:45.763823 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.763787 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:45.763872 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:45.763836 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:46.778355 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:46.777630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" event={"ID":"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f","Type":"ContainerStarted","Data":"f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663"} Jul 10 21:20:47.501438 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:47.501406 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:47.501611 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:47.501505 2577 patch_prober.go:28] interesting pod/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 10 21:20:47.501611 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:47.501538 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 10 21:20:48.501815 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:48.501775 2577 patch_prober.go:28] interesting pod/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 10 21:20:48.502175 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:48.501836 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 10 21:20:49.502313 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:49.502270 2577 patch_prober.go:28] interesting pod/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 10 21:20:49.502750 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:49.502346 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 10 21:20:50.502088 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:50.502052 2577 patch_prober.go:28] interesting pod/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 10 21:20:50.502259 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:50.502112 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 10 21:20:50.662293 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:50.662262 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:20:50.681090 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:50.681056 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:50.780953 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:50.780884 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:51.505152 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.505126 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:51.505546 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.505527 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:51.506073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.506057 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:20:51.808407 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.808375 2577 generic.go:356] "Generic (PLEG): container finished" podID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerID="28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09" exitCode=0 Jul 10 21:20:51.808812 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.808454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" event={"ID":"6e1de2a1-9499-4a31-9cf3-d450524bc4db","Type":"ContainerDied","Data":"28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09"} Jul 10 21:20:51.809045 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.809022 2577 scope.go:117] "RemoveContainer" containerID="28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09" Jul 10 21:20:51.810019 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.809991 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:51.810086 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.810069 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:51.810176 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:51.810118 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:20:52.814846 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:52.814800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" event={"ID":"6e1de2a1-9499-4a31-9cf3-d450524bc4db","Type":"ContainerStarted","Data":"c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4"} Jul 10 21:20:52.872088 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:52.872054 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:52.972932 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:52.972898 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:20:53.232050 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:53.231945 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:53.233177 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:53.233141 2577 patch_prober.go:28] interesting pod/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 10 21:20:53.233323 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:53.233216 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 10 21:20:53.718946 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:53.718907 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:54.231789 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:54.231748 2577 patch_prober.go:28] interesting pod/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 10 21:20:54.232268 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:54.231820 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 10 21:20:55.231381 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:55.231339 2577 patch_prober.go:28] interesting pod/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 10 21:20:55.231605 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:55.231401 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 10 21:20:56.231372 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:56.231327 2577 patch_prober.go:28] interesting pod/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 10 21:20:56.231882 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:56.231404 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 10 21:20:57.235290 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:57.235256 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:20:57.236215 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:20:57.236179 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:21:01.316625 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:01.316591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:01.416411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:01.416381 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:02.768929 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:02.768891 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:02.870893 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:02.870866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:03.045221 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:03.045144 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:06.552169 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:06.552137 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:21:07.322521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:07.322482 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:13.144873 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:13.144838 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:13.247426 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:13.247393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:18.046932 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:18.046896 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2"] Jul 10 21:21:18.047410 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:18.047153 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" containerID="cri-o://f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663" gracePeriod=30 Jul 10 21:21:18.073607 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:18.073577 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:18.312357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:18.312269 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:19.043145 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:19.043111 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:19.144069 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:19.144037 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:21.507971 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:21.507932 2577 patch_prober.go:28] interesting pod/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:21:21.508441 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:21.507991 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:21:23.296370 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.296349 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:21:23.424573 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424479 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-data\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.424573 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424553 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-certs\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.424841 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424598 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istiod-ca-cert\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.424841 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424626 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-envoy\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.424841 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424721 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfh5n\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-kube-api-access-bfh5n\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.424841 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424788 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-token\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.424841 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424819 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-credential-socket\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.425096 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424856 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-socket\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.425096 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.424896 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-podinfo\") pod \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\" (UID: \"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f\") " Jul 10 21:21:23.425096 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425031 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:23.425243 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425207 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:23.425316 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425289 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:21:23.425370 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425321 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.425370 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425338 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.425472 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425419 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:23.425633 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.425610 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:23.427133 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.427101 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-kube-api-access-bfh5n" (OuterVolumeSpecName: "kube-api-access-bfh5n") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "kube-api-access-bfh5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:21:23.427133 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.427124 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:23.427443 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.427409 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:21:23.427509 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.427454 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" (UID: "2ad01fc8-62eb-4df2-b905-f1f3c1fb667f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:21:23.526105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526066 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.526105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526098 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.526105 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526111 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.526337 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526119 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfh5n\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-kube-api-access-bfh5n\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.526337 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526129 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.526337 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526138 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.526337 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.526146 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:23.969621 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.969588 2577 generic.go:356] "Generic (PLEG): container finished" podID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerID="f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663" exitCode=0 Jul 10 21:21:23.969851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.969639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" event={"ID":"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f","Type":"ContainerDied","Data":"f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663"} Jul 10 21:21:23.969851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.969672 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" Jul 10 21:21:23.969851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.969733 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2" event={"ID":"2ad01fc8-62eb-4df2-b905-f1f3c1fb667f","Type":"ContainerDied","Data":"c18e23864ffe3e947b54102ce4d4c93084c371baff28368c3ee787473db7bf6c"} Jul 10 21:21:23.969851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.969758 2577 scope.go:117] "RemoveContainer" containerID="f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663" Jul 10 21:21:23.979235 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.979217 2577 scope.go:117] "RemoveContainer" containerID="34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727" Jul 10 21:21:23.987067 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.987042 2577 scope.go:117] "RemoveContainer" containerID="f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663" Jul 10 21:21:23.987339 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:21:23.987320 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663\": container with ID starting with f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663 not found: ID does not exist" containerID="f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663" Jul 10 21:21:23.987391 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.987347 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663"} err="failed to get container status \"f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663\": rpc error: code = NotFound desc = could not find container \"f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663\": container with ID starting with f15f88cf4cc7db2d8f642cc5c4e9143983628bb3e4dd350545a1ed06fcf57663 not found: ID does not exist" Jul 10 21:21:23.987391 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.987365 2577 scope.go:117] "RemoveContainer" containerID="34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727" Jul 10 21:21:23.987601 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:21:23.987584 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727\": container with ID starting with 34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727 not found: ID does not exist" containerID="34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727" Jul 10 21:21:23.987666 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.987606 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727"} err="failed to get container status \"34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727\": rpc error: code = NotFound desc = could not find container \"34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727\": container with ID starting with 34810ab39544e3c54845b8da3785075126b0ad1f05cb381dd4e52a9c52e83727 not found: ID does not exist" Jul 10 21:21:23.993405 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.993382 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2"] Jul 10 21:21:23.998958 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:23.998937 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ed1347-openshift-default-dc4d96887-nh2q2"] Jul 10 21:21:25.165609 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:25.165580 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" path="/var/lib/kubelet/pods/2ad01fc8-62eb-4df2-b905-f1f3c1fb667f/volumes" Jul 10 21:21:28.934233 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:28.934198 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:32.153468 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:32.153431 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:21:33.539645 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:33.539611 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:39.218771 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:39.218738 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:39.321347 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:39.321304 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:44.208205 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:44.208167 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn"] Jul 10 21:21:44.208607 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:44.208411 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" containerID="cri-o://c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4" gracePeriod=30 Jul 10 21:21:44.232426 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:44.232398 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:44.459876 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:44.459790 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:44.822594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:44.822562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:45.113299 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113211 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx"] Jul 10 21:21:45.113649 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113631 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" Jul 10 21:21:45.113763 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113672 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" Jul 10 21:21:45.113763 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113706 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" Jul 10 21:21:45.113763 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113715 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" Jul 10 21:21:45.113919 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113807 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" Jul 10 21:21:45.113919 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113821 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" Jul 10 21:21:45.113919 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.113834 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dae56460-d99f-43d0-9500-5bfdbeadca8f" containerName="istio-proxy" Jul 10 21:21:45.118626 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.118603 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.121766 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.121742 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-920ed7-openshift-default-dockercfg-s5988\"" Jul 10 21:21:45.130960 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.130939 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx"] Jul 10 21:21:45.157864 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.157838 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:45.196646 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-socket\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.196780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istiod-ca-cert\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.196780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb7xp\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-kube-api-access-sb7xp\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.196780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-envoy\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.196780 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-data\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.197010 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-podinfo\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.197010 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-token\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.197010 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.196998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-certs\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.197151 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.197047 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-credential-socket\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298427 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb7xp\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-kube-api-access-sb7xp\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298427 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-envoy\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-data\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-podinfo\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-token\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-certs\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-credential-socket\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298849 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-socket\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istiod-ca-cert\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.298978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.298932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-data\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.299425 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.299235 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-socket\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.299523 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.299494 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-credential-socket\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.299800 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.299773 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-certs\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.299930 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.299785 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istiod-ca-cert\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.300933 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.300915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-envoy\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.301003 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.300950 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-podinfo\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.306701 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.306681 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb7xp\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-kube-api-access-sb7xp\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.306846 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.306826 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-token\") pod \"e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.431345 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.431304 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:45.510923 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.510893 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:45.576468 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.576436 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx"] Jul 10 21:21:45.578403 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:21:45.578379 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca42396_f3d8_4b06_b2bb_748636d87c1a.slice/crio-139e50d2a3ba332eefdad01d731cd00133fda2df52f67d3922c467b6980556ef WatchSource:0}: Error finding container 139e50d2a3ba332eefdad01d731cd00133fda2df52f67d3922c467b6980556ef: Status 404 returned error can't find the container with id 139e50d2a3ba332eefdad01d731cd00133fda2df52f67d3922c467b6980556ef Jul 10 21:21:45.580532 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.580486 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:21:45.580771 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.580577 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:21:45.580771 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:45.580626 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:21:46.086341 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:46.086302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" event={"ID":"4ca42396-f3d8-4b06-b2bb-748636d87c1a","Type":"ContainerStarted","Data":"68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6"} Jul 10 21:21:46.086517 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:46.086368 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" event={"ID":"4ca42396-f3d8-4b06-b2bb-748636d87c1a","Type":"ContainerStarted","Data":"139e50d2a3ba332eefdad01d731cd00133fda2df52f67d3922c467b6980556ef"} Jul 10 21:21:46.107062 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:46.107012 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podStartSLOduration=1.10699834 podStartE2EDuration="1.10699834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:21:46.104438525 +0000 UTC m=+2641.452747855" watchObservedRunningTime="2026-07-10 21:21:46.10699834 +0000 UTC m=+2641.455307671" Jul 10 21:21:46.431554 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:46.431480 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:46.435938 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:46.435917 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:47.093400 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:47.093369 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:47.094570 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:47.094545 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:21:48.091014 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:48.090984 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:48.196499 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:48.196467 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:48.287596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:48.287562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:48.394365 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:48.394276 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:48.845074 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:48.845038 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:48.945041 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:48.945008 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:49.466777 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.466749 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:21:49.538697 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538640 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcd6j\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-kube-api-access-kcd6j\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.538856 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538719 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-socket\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.538856 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-data\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.538944 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-credential-socket\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.538944 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538918 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-podinfo\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.539055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538959 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:49.539055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.538978 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istiod-ca-cert\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.539055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539009 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-envoy\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.539055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539045 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-certs\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.539260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539075 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-token\") pod \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\" (UID: \"6e1de2a1-9499-4a31-9cf3-d450524bc4db\") " Jul 10 21:21:49.539260 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539192 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:49.539357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539310 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:21:49.539512 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539427 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.539603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539509 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:49.539603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539523 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.539603 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539540 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.539754 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.539635 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:49.541007 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.540984 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-kube-api-access-kcd6j" (OuterVolumeSpecName: "kube-api-access-kcd6j") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "kube-api-access-kcd6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:21:49.541220 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.541206 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:21:49.541429 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.541411 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:21:49.541495 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.541423 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e1de2a1-9499-4a31-9cf3-d450524bc4db" (UID: "6e1de2a1-9499-4a31-9cf3-d450524bc4db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:21:49.640663 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.640634 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.640779 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.640681 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.640779 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.640692 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.640779 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.640700 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcd6j\" (UniqueName: \"kubernetes.io/projected/6e1de2a1-9499-4a31-9cf3-d450524bc4db-kube-api-access-kcd6j\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.640779 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.640708 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:49.640779 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:49.640716 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1de2a1-9499-4a31-9cf3-d450524bc4db-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:21:50.108902 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.108863 2577 generic.go:356] "Generic (PLEG): container finished" podID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerID="c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4" exitCode=0 Jul 10 21:21:50.109068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.108918 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" Jul 10 21:21:50.109068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.108951 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" event={"ID":"6e1de2a1-9499-4a31-9cf3-d450524bc4db","Type":"ContainerDied","Data":"c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4"} Jul 10 21:21:50.109068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.109009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn" event={"ID":"6e1de2a1-9499-4a31-9cf3-d450524bc4db","Type":"ContainerDied","Data":"413070a3c329f20a2a9206aac43802f06f5f131f1f44c7ddbb22a5325e386d40"} Jul 10 21:21:50.109068 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.109031 2577 scope.go:117] "RemoveContainer" containerID="c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4" Jul 10 21:21:50.118371 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.118350 2577 scope.go:117] "RemoveContainer" containerID="28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09" Jul 10 21:21:50.126019 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.126002 2577 scope.go:117] "RemoveContainer" containerID="c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4" Jul 10 21:21:50.126252 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:21:50.126232 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4\": container with ID starting with c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4 not found: ID does not exist" containerID="c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4" Jul 10 21:21:50.126314 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.126258 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4"} err="failed to get container status \"c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4\": rpc error: code = NotFound desc = could not find container \"c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4\": container with ID starting with c9dc6bf2f525b65804d68dbc519c35609cd3bedbcc32b142732c05f2563f60a4 not found: ID does not exist" Jul 10 21:21:50.126314 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.126273 2577 scope.go:117] "RemoveContainer" containerID="28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09" Jul 10 21:21:50.126511 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:21:50.126490 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09\": container with ID starting with 28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09 not found: ID does not exist" containerID="28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09" Jul 10 21:21:50.126555 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.126518 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09"} err="failed to get container status \"28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09\": rpc error: code = NotFound desc = could not find container \"28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09\": container with ID starting with 28ca4c0bff344896ffbbb6bbadc35cd8457abe8ac897ff7c9f66cc5a879d5f09 not found: ID does not exist" Jul 10 21:21:50.131717 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.131695 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn"] Jul 10 21:21:50.134791 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:50.134769 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c4ebda-openshift-default-7f4bd9db59-2kxdn"] Jul 10 21:21:51.164636 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:51.164605 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" path="/var/lib/kubelet/pods/6e1de2a1-9499-4a31-9cf3-d450524bc4db/volumes" Jul 10 21:21:52.593411 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:52.593376 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:52.943637 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:52.943554 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:53.235724 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.235628 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v"] Jul 10 21:21:53.236191 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236173 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" Jul 10 21:21:53.236259 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236195 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad01fc8-62eb-4df2-b905-f1f3c1fb667f" containerName="istio-proxy" Jul 10 21:21:53.236259 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236220 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" Jul 10 21:21:53.236259 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236248 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" Jul 10 21:21:53.236421 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236283 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" Jul 10 21:21:53.236421 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236292 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" Jul 10 21:21:53.236421 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.236365 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" Jul 10 21:21:53.239687 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.239643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.245071 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.245053 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-6e676e-openshift-default-dockercfg-4cf6k\"" Jul 10 21:21:53.254694 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.254670 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v"] Jul 10 21:21:53.272007 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.271980 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:53.375594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-socket\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375594 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375592 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-certs\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65k9g\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-kube-api-access-65k9g\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-envoy\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istiod-ca-cert\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375781 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-data\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-credential-socket\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.375835 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-podinfo\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.376037 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.375866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-token\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476374 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-socket\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476374 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-certs\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65k9g\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-kube-api-access-65k9g\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-envoy\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istiod-ca-cert\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476529 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-data\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-credential-socket\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-podinfo\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476629 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-token\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476825 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-certs\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.476994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.476916 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-socket\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.477070 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.477041 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-credential-socket\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.477155 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.477132 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-data\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.477362 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.477341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istiod-ca-cert\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.478864 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.478836 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-envoy\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.479121 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.479105 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-podinfo\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.485268 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.485247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-token\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.485544 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.485523 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65k9g\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-kube-api-access-65k9g\") pod \"e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.550523 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.550496 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:53.600469 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.600432 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:53.690577 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.690552 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v"] Jul 10 21:21:53.692640 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:21:53.692603 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa7ab78_cc5a_48cc_b84e_8ab07516ca56.slice/crio-53c3b63e03d83d99321a67b256a9b5f7657429d0eefe8fd856455107e3f22db4 WatchSource:0}: Error finding container 53c3b63e03d83d99321a67b256a9b5f7657429d0eefe8fd856455107e3f22db4: Status 404 returned error can't find the container with id 53c3b63e03d83d99321a67b256a9b5f7657429d0eefe8fd856455107e3f22db4 Jul 10 21:21:53.695207 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.695176 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:21:53.695312 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.695280 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:21:53.695375 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.695327 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:21:53.955371 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:53.955284 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:21:54.135997 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:54.135952 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" event={"ID":"efa7ab78-cc5a-48cc-b84e-8ab07516ca56","Type":"ContainerStarted","Data":"c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8"} Jul 10 21:21:54.136278 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:54.136247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" event={"ID":"efa7ab78-cc5a-48cc-b84e-8ab07516ca56","Type":"ContainerStarted","Data":"53c3b63e03d83d99321a67b256a9b5f7657429d0eefe8fd856455107e3f22db4"} Jul 10 21:21:54.160911 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:54.160860 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" podStartSLOduration=1.160844926 podStartE2EDuration="1.160844926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:21:54.158606337 +0000 UTC m=+2649.506915668" watchObservedRunningTime="2026-07-10 21:21:54.160844926 +0000 UTC m=+2649.509154256" Jul 10 21:21:54.550834 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:54.550801 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:54.555906 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:54.555880 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:55.145216 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:55.145179 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:55.150515 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:55.150478 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:21:56.099057 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:56.099022 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:56.179354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:56.179318 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:56.255006 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:56.254962 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:56.511203 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:56.511170 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:56.873261 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:56.873178 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:21:56.972284 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:21:56.972254 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:00.621062 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:00.621024 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:02.206071 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:02.205265 2577 patch_prober.go:28] interesting pod/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": read tcp 10.133.0.2:33804->10.133.0.47:15021: read: connection reset by peer" start-of-body= Jul 10 21:22:02.206071 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:02.205327 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": read tcp 10.133.0.2:33804->10.133.0.47:15021: read: connection reset by peer" Jul 10 21:22:03.192080 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:03.192049 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerID="68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6" exitCode=0 Jul 10 21:22:03.192251 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:03.192124 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" event={"ID":"4ca42396-f3d8-4b06-b2bb-748636d87c1a","Type":"ContainerDied","Data":"68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6"} Jul 10 21:22:03.192560 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:03.192547 2577 scope.go:117] "RemoveContainer" containerID="68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6" Jul 10 21:22:03.193612 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:03.193581 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:22:03.193705 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:03.193689 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:22:03.193750 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:03.193733 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:22:04.199157 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:04.199118 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" event={"ID":"4ca42396-f3d8-4b06-b2bb-748636d87c1a","Type":"ContainerStarted","Data":"ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048"} Jul 10 21:22:04.431710 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:04.431641 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:22:04.431999 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:04.431969 2577 patch_prober.go:28] interesting pod/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 21:22:04.432102 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:04.432025 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 21:22:05.432003 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:05.431960 2577 patch_prober.go:28] interesting pod/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 21:22:05.432567 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:05.432535 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 21:22:05.746505 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:05.746437 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:06.154323 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:06.154219 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:22:06.432994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:06.432410 2577 patch_prober.go:28] interesting pod/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 21:22:06.432994 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:06.432484 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 21:22:07.432310 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:07.432271 2577 patch_prober.go:28] interesting pod/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 21:22:07.432609 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:07.432579 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 21:22:08.435848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:08.435819 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:22:08.436350 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:08.436218 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:22:08.436961 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:08.436940 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:22:09.229808 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.229778 2577 generic.go:356] "Generic (PLEG): container finished" podID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerID="c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8" exitCode=0 Jul 10 21:22:09.229978 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.229846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" event={"ID":"efa7ab78-cc5a-48cc-b84e-8ab07516ca56","Type":"ContainerDied","Data":"c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8"} Jul 10 21:22:09.230312 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.230298 2577 scope.go:117] "RemoveContainer" containerID="c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8" Jul 10 21:22:09.231221 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.231199 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:22:09.231289 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.231265 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:22:09.231327 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.231295 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:22:09.819460 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.819417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:09.922034 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:09.922002 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:10.145644 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:10.145492 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:22:10.244097 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:10.244058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" event={"ID":"efa7ab78-cc5a-48cc-b84e-8ab07516ca56","Type":"ContainerStarted","Data":"cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f"} Jul 10 21:22:10.553059 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:10.553007 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:22:10.553381 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:10.553201 2577 patch_prober.go:28] interesting pod/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 21:22:10.553381 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:10.553276 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 21:22:11.551383 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:11.551332 2577 patch_prober.go:28] interesting pod/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 21:22:11.551911 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:11.551419 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 21:22:12.551445 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:12.551395 2577 patch_prober.go:28] interesting pod/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 21:22:12.552021 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:12.551465 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 21:22:13.551565 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:13.551123 2577 patch_prober.go:28] interesting pod/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 21:22:13.551565 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:13.551188 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 21:22:14.554813 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:14.554777 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:22:14.555227 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:14.555080 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:22:14.555848 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:14.555828 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:22:17.681801 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:17.681763 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:17.784170 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:17.784134 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:18.644163 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:18.644134 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:22.251543 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:22.251502 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:22:25.070649 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:25.070616 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:45.514214 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:45.514176 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:22:49.057054 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:22:49.057015 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:23:03.992488 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:23:03.992456 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:23:24.444162 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:23:24.444123 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:23:40.553318 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:23:40.553276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:23:59.344942 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:23:59.344904 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:24:16.352077 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:24:16.351996 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:24:31.850611 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:24:31.850577 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:24:42.782430 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:24:42.782399 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:24:43.594211 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:24:43.594174 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:24:47.648135 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:24:47.648080 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:24:53.206610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:24:53.206568 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:03.855282 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:03.855249 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:25:19.177073 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:19.177036 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:24.053299 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:24.053265 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:25:28.463735 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:28.463693 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:28.520904 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:28.520871 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:31.243793 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:31.243758 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:38.347521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:38.347437 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:25:51.536562 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:51.536528 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:51.636578 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:51.636536 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:25:59.453249 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:25:59.453217 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:26:04.854042 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:04.854005 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:26:05.698282 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:05.698248 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:26:10.681382 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:10.681342 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:26:18.352810 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:18.352776 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:26:19.870306 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:19.870269 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:26:19.970777 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:19.970735 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:26:38.003561 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:38.003503 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:26:42.359492 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:42.359457 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:26:51.398254 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:26:51.398221 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:03.853041 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:03.853005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:27:10.812936 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:10.812898 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:10.912015 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:10.911983 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:18.469017 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:18.468959 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:18.571225 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:18.571186 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:19.260189 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:19.260156 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:27:26.291927 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:26.291894 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:27.083718 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:27.083680 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:33.644596 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:33.644558 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:27:46.764470 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:46.764416 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:46.863506 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:46.863464 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:27:56.772722 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:27:56.772688 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:00.357851 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:00.357812 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:28:01.360316 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:01.360279 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:07.390706 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:07.390669 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:07.491116 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:07.491082 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:12.040912 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:12.040879 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx"] Jul 10 21:28:12.041318 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:12.041101 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" containerID="cri-o://ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048" gracePeriod=30 Jul 10 21:28:12.062027 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:12.061986 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:12.305492 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:12.305391 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:12.947971 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:12.947938 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:13.048422 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:13.048382 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:17.285101 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.285066 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:28:17.329521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329450 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-certs\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329485 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-credential-socket\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329521 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329521 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-data\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329829 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329537 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-envoy\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329829 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329580 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-token\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329829 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329597 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istiod-ca-cert\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329829 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329614 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb7xp\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-kube-api-access-sb7xp\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329829 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329648 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-podinfo\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.329829 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329710 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-socket\") pod \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\" (UID: \"4ca42396-f3d8-4b06-b2bb-748636d87c1a\") " Jul 10 21:28:17.330135 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.329898 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:17.330135 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.330020 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.330410 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.330383 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:28:17.330512 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.330461 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:17.330512 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.330490 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:17.331281 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.331250 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:17.332107 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.332067 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:28:17.332207 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.332154 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-kube-api-access-sb7xp" (OuterVolumeSpecName: "kube-api-access-sb7xp") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "kube-api-access-sb7xp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:28:17.332294 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.332277 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:17.332388 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.332374 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ca42396-f3d8-4b06-b2bb-748636d87c1a" (UID: "4ca42396-f3d8-4b06-b2bb-748636d87c1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:28:17.431490 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431451 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431490 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431479 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431490 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431490 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431490 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431498 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431922 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431505 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431922 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431512 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb7xp\" (UniqueName: \"kubernetes.io/projected/4ca42396-f3d8-4b06-b2bb-748636d87c1a-kube-api-access-sb7xp\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431922 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431522 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca42396-f3d8-4b06-b2bb-748636d87c1a-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:17.431922 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:17.431529 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca42396-f3d8-4b06-b2bb-748636d87c1a-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:18.052964 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.052933 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerID="ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048" exitCode=0 Jul 10 21:28:18.053143 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.052967 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" event={"ID":"4ca42396-f3d8-4b06-b2bb-748636d87c1a","Type":"ContainerDied","Data":"ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048"} Jul 10 21:28:18.053143 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.053025 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" event={"ID":"4ca42396-f3d8-4b06-b2bb-748636d87c1a","Type":"ContainerDied","Data":"139e50d2a3ba332eefdad01d731cd00133fda2df52f67d3922c467b6980556ef"} Jul 10 21:28:18.053143 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.052980 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx" Jul 10 21:28:18.053143 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.053053 2577 scope.go:117] "RemoveContainer" containerID="ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048" Jul 10 21:28:18.064400 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.064381 2577 scope.go:117] "RemoveContainer" containerID="68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6" Jul 10 21:28:18.072288 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.072266 2577 scope.go:117] "RemoveContainer" containerID="ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048" Jul 10 21:28:18.072524 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:28:18.072508 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048\": container with ID starting with ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048 not found: ID does not exist" containerID="ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048" Jul 10 21:28:18.072579 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.072532 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048"} err="failed to get container status \"ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048\": rpc error: code = NotFound desc = could not find container \"ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048\": container with ID starting with ec6a1bdda7a4dc42be492c2281210658649bf27cb68857af2375f37e530bb048 not found: ID does not exist" Jul 10 21:28:18.072579 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.072548 2577 scope.go:117] "RemoveContainer" containerID="68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6" Jul 10 21:28:18.072831 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:28:18.072813 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6\": container with ID starting with 68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6 not found: ID does not exist" containerID="68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6" Jul 10 21:28:18.072878 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.072837 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6"} err="failed to get container status \"68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6\": rpc error: code = NotFound desc = could not find container \"68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6\": container with ID starting with 68e1626ff5e5c646625538c52da530c33f2a3bd44731f7107edf12522b5820b6 not found: ID does not exist" Jul 10 21:28:18.076400 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.076378 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx"] Jul 10 21:28:18.080643 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:18.080619 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-920ed7-openshift-default-67bf556d95-g2fnx"] Jul 10 21:28:19.163948 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:19.163920 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" path="/var/lib/kubelet/pods/4ca42396-f3d8-4b06-b2bb-748636d87c1a/volumes" Jul 10 21:28:23.129597 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:23.129560 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:26.152511 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:26.152470 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-c6gdk"] Jul 10 21:28:27.527535 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:27.527500 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:33.632375 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:33.632339 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:33.733677 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:33.733621 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:38.205684 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:38.205594 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v"] Jul 10 21:28:38.206055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:38.205851 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" containerID="cri-o://cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f" gracePeriod=30 Jul 10 21:28:38.228339 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:38.228293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:38.464342 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:38.464258 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:43.456318 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.456296 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:28:43.459354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459333 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istiod-ca-cert\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459363 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-podinfo\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459389 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-data\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459827 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459438 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-token\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459827 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459468 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-socket\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459827 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459518 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-credential-socket\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459827 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459585 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-certs\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459827 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459613 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65k9g\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-kube-api-access-65k9g\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.459827 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459646 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-envoy\") pod \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\" (UID: \"efa7ab78-cc5a-48cc-b84e-8ab07516ca56\") " Jul 10 21:28:43.460168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459646 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:28:43.460168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459790 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:43.460168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.459993 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:43.460299 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.460188 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:43.460299 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.460253 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-data" (OuterVolumeSpecName: "istio-data") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:43.461574 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.461549 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:28:43.461923 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.461902 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-token" (OuterVolumeSpecName: "istio-token") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:28:43.461985 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.461940 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-kube-api-access-65k9g" (OuterVolumeSpecName: "kube-api-access-65k9g") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "kube-api-access-65k9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:28:43.462055 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.462034 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efa7ab78-cc5a-48cc-b84e-8ab07516ca56" (UID: "efa7ab78-cc5a-48cc-b84e-8ab07516ca56"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:28:43.560472 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560442 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-certs\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560472 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560469 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65k9g\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-kube-api-access-65k9g\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560472 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560479 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-envoy\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560698 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560487 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istiod-ca-cert\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560698 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560495 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-podinfo\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560698 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560503 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-data\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560698 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560510 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-istio-token\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560698 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560519 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-workload-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:43.560698 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:43.560527 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa7ab78-cc5a-48cc-b84e-8ab07516ca56-credential-socket\") on node \"ip-10-0-136-104.ec2.internal\" DevicePath \"\"" Jul 10 21:28:44.180189 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.180154 2577 generic.go:356] "Generic (PLEG): container finished" podID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerID="cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f" exitCode=0 Jul 10 21:28:44.180357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.180192 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" Jul 10 21:28:44.180357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.180202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" event={"ID":"efa7ab78-cc5a-48cc-b84e-8ab07516ca56","Type":"ContainerDied","Data":"cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f"} Jul 10 21:28:44.180357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.180247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v" event={"ID":"efa7ab78-cc5a-48cc-b84e-8ab07516ca56","Type":"ContainerDied","Data":"53c3b63e03d83d99321a67b256a9b5f7657429d0eefe8fd856455107e3f22db4"} Jul 10 21:28:44.180357 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.180271 2577 scope.go:117] "RemoveContainer" containerID="cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f" Jul 10 21:28:44.189129 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.189112 2577 scope.go:117] "RemoveContainer" containerID="c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8" Jul 10 21:28:44.196863 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.196844 2577 scope.go:117] "RemoveContainer" containerID="cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f" Jul 10 21:28:44.197102 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:28:44.197081 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f\": container with ID starting with cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f not found: ID does not exist" containerID="cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f" Jul 10 21:28:44.197165 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.197108 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f"} err="failed to get container status \"cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f\": rpc error: code = NotFound desc = could not find container \"cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f\": container with ID starting with cc23ecbe9759b0ddcc7070ae5d9b82396e31aa8904e2fea934ea56260f59b25f not found: ID does not exist" Jul 10 21:28:44.197165 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.197125 2577 scope.go:117] "RemoveContainer" containerID="c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8" Jul 10 21:28:44.197349 ip-10-0-136-104 kubenswrapper[2577]: E0710 21:28:44.197333 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8\": container with ID starting with c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8 not found: ID does not exist" containerID="c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8" Jul 10 21:28:44.197387 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.197354 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8"} err="failed to get container status \"c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8\": rpc error: code = NotFound desc = could not find container \"c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8\": container with ID starting with c5ec1f0f9d1e62d32828c7a9313f6045d75b0ab07d654d06ef97b02f72ca3ba8 not found: ID does not exist" Jul 10 21:28:44.203093 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.203072 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v"] Jul 10 21:28:44.207217 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.207192 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-6e676e-openshift-default-585985dc-9tm6v"] Jul 10 21:28:44.526042 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.526024 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-m6g28_3061a25c-4747-4ff0-98b4-1a8776efc82f/copy-manifests/0.log" Jul 10 21:28:44.533847 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:44.533827 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-m6g28_3061a25c-4747-4ff0-98b4-1a8776efc82f/manager/0.log" Jul 10 21:28:45.164539 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:45.164507 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" path="/var/lib/kubelet/pods/efa7ab78-cc5a-48cc-b84e-8ab07516ca56/volumes" Jul 10 21:28:45.181306 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:45.181285 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29728620-t7ntl_b02547c5-1bd6-44f5-935f-f0abee814f83/cleanup/0.log" Jul 10 21:28:45.422910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:45.422845 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-35794783-jppph_7531a691-9c2d-4b88-8567-2f4207708147/revoke-keys/0.log" Jul 10 21:28:45.542020 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:45.541996 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-6bb25bb7-6d7tk_6f210d3b-dbbe-4709-9321-3e42dd8f4c12/revoke-keys/0.log" Jul 10 21:28:45.876819 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:45.876792 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-183404a5-2cjm9_78f293be-0034-4e22-b9c4-3547a423c803/revoke-keys/0.log" Jul 10 21:28:45.986060 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:45.986029 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-42c194-ht5jv_356a7100-5b05-443e-9c98-2f10161ed2f4/revoke-keys/0.log" Jul 10 21:28:46.319856 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:46.319832 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-46c4a127-rkglm_5f5b2f9f-2f39-4cb7-9b75-7bbfba21e32e/revoke-keys/0.log" Jul 10 21:28:48.753232 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:48.753185 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-c6gdk_b8ee921b-aaca-4826-a64e-384f62d4724a/limitador/0.log" Jul 10 21:28:49.231155 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:49.231070 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qn694_32575539-5873-49c4-8da6-9e1aba76c3a6/istio-proxy/0.log" Jul 10 21:28:49.353861 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:49.353832 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nl4qq_daca5d24-2543-4659-bd4f-571d9e41e545/discovery/0.log" Jul 10 21:28:49.697825 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:49.697794 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/1.log" Jul 10 21:28:50.067634 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:50.067590 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69db686bdd-kc8rg_fbafa86c-d243-443d-bfcd-c78189d42283/router/0.log" Jul 10 21:28:55.162039 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:55.162006 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:55.256966 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:55.256932 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" limit=1024 Jul 10 21:28:57.943141 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:57.943118 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nbn2w_decac6e6-5885-4081-8f10-041e44c6a00e/global-pull-secret-syncer/0.log" Jul 10 21:28:58.072606 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:58.072571 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jvmm8_9f059320-52a4-4f1a-b987-0a60ed0159d7/konnectivity-agent/0.log" Jul 10 21:28:58.171942 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:28:58.171917 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-104.ec2.internal_d8de27905a457ddc122bd67be75fc31e/haproxy/0.log" Jul 10 21:29:03.632949 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:03.632915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-c6gdk_b8ee921b-aaca-4826-a64e-384f62d4724a/limitador/0.log" Jul 10 21:29:05.665909 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.665883 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-bf5b4_e4d030b2-d55b-439f-81dd-04961ec3993b/cluster-monitoring-operator/0.log" Jul 10 21:29:05.695506 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.695479 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wtjxf_db15070c-b382-462b-b6e3-db263e7227b7/kube-state-metrics/0.log" Jul 10 21:29:05.719932 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.719906 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wtjxf_db15070c-b382-462b-b6e3-db263e7227b7/kube-rbac-proxy-main/0.log" Jul 10 21:29:05.738261 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.738239 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wtjxf_db15070c-b382-462b-b6e3-db263e7227b7/kube-rbac-proxy-self/0.log" Jul 10 21:29:05.767631 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.767604 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fcf7d76f4-blhvm_1f9d9ea2-2176-4139-9324-2e04b956bb5a/metrics-server/0.log" Jul 10 21:29:05.963489 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.963407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7txr_148fe200-6d1d-4a1d-884a-92435c9726bb/node-exporter/0.log" Jul 10 21:29:05.983407 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:05.983368 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7txr_148fe200-6d1d-4a1d-884a-92435c9726bb/kube-rbac-proxy/0.log" Jul 10 21:29:06.002368 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.002340 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7txr_148fe200-6d1d-4a1d-884a-92435c9726bb/init-textfile/0.log" Jul 10 21:29:06.027223 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.027199 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-n52fz_60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f/kube-rbac-proxy-main/0.log" Jul 10 21:29:06.045262 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.045236 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-n52fz_60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f/kube-rbac-proxy-self/0.log" Jul 10 21:29:06.063541 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.063518 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-n52fz_60fb79fb-6e7c-4eb3-b009-32cfbfd3f10f/openshift-state-metrics/0.log" Jul 10 21:29:06.098921 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.098890 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/prometheus/0.log" Jul 10 21:29:06.121549 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.121517 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/config-reloader/0.log" Jul 10 21:29:06.140890 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.140865 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/thanos-sidecar/0.log" Jul 10 21:29:06.161914 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.161891 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/kube-rbac-proxy-web/0.log" Jul 10 21:29:06.192971 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.192945 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/kube-rbac-proxy/0.log" Jul 10 21:29:06.216395 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.216303 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/kube-rbac-proxy-thanos/0.log" Jul 10 21:29:06.235966 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.235933 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a668e185-3055-4ffa-95d4-055ca09a78b6/init-config-reloader/0.log" Jul 10 21:29:06.305124 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:06.305095 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-g98lj_922c7a96-bf9b-4ead-904a-683097ec926a/prometheus-operator-admission-webhook/0.log" Jul 10 21:29:08.176667 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.176630 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch"] Jul 10 21:29:08.177163 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177133 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" Jul 10 21:29:08.177163 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177152 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" Jul 10 21:29:08.177287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177175 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" Jul 10 21:29:08.177287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177183 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" Jul 10 21:29:08.177287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177204 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" Jul 10 21:29:08.177287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177214 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" Jul 10 21:29:08.177287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177228 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" Jul 10 21:29:08.177287 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177235 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" Jul 10 21:29:08.177610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177316 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" Jul 10 21:29:08.177610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177328 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" Jul 10 21:29:08.177610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177337 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa7ab78-cc5a-48cc-b84e-8ab07516ca56" containerName="istio-proxy" Jul 10 21:29:08.177610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177347 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1de2a1-9499-4a31-9cf3-d450524bc4db" containerName="istio-proxy" Jul 10 21:29:08.177610 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.177357 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca42396-f3d8-4b06-b2bb-748636d87c1a" containerName="istio-proxy" Jul 10 21:29:08.180571 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.180551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.183148 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.183126 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bhd5q\"/\"kube-root-ca.crt\"" Jul 10 21:29:08.184149 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.184133 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bhd5q\"/\"default-dockercfg-blt7c\"" Jul 10 21:29:08.184229 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.184149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bhd5q\"/\"openshift-service-ca.crt\"" Jul 10 21:29:08.189353 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.189328 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch"] Jul 10 21:29:08.280056 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.280021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84xq8\" (UniqueName: \"kubernetes.io/projected/c0538b60-2cfc-4874-aa79-1b65b51c327d-kube-api-access-84xq8\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.280056 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.280059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-proc\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.280295 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.280218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-podres\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.280353 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.280328 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-sys\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.280440 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.280424 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-lib-modules\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.380948 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.380910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84xq8\" (UniqueName: \"kubernetes.io/projected/c0538b60-2cfc-4874-aa79-1b65b51c327d-kube-api-access-84xq8\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.380948 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.380950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-proc\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-podres\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-sys\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-lib-modules\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381168 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381087 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-proc\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381185 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-sys\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381275 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-lib-modules\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.381354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.381280 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c0538b60-2cfc-4874-aa79-1b65b51c327d-podres\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.389026 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.389002 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84xq8\" (UniqueName: \"kubernetes.io/projected/c0538b60-2cfc-4874-aa79-1b65b51c327d-kube-api-access-84xq8\") pod \"perf-node-gather-daemonset-ktlch\" (UID: \"c0538b60-2cfc-4874-aa79-1b65b51c327d\") " pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.491689 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.491627 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:08.625451 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:08.625412 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch"] Jul 10 21:29:08.628871 ip-10-0-136-104 kubenswrapper[2577]: W0710 21:29:08.628825 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc0538b60_2cfc_4874_aa79_1b65b51c327d.slice/crio-341b2dba1002264bf21a9b0f813c42e78a38f4478cab3da8b549debf6b17ae6c WatchSource:0}: Error finding container 341b2dba1002264bf21a9b0f813c42e78a38f4478cab3da8b549debf6b17ae6c: Status 404 returned error can't find the container with id 341b2dba1002264bf21a9b0f813c42e78a38f4478cab3da8b549debf6b17ae6c Jul 10 21:29:09.297558 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:09.297524 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" event={"ID":"c0538b60-2cfc-4874-aa79-1b65b51c327d","Type":"ContainerStarted","Data":"059480305b5da2ca714adb6274ec3ecaf327e7f0c6e768294133b7d38ed88523"} Jul 10 21:29:09.297965 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:09.297579 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" event={"ID":"c0538b60-2cfc-4874-aa79-1b65b51c327d","Type":"ContainerStarted","Data":"341b2dba1002264bf21a9b0f813c42e78a38f4478cab3da8b549debf6b17ae6c"} Jul 10 21:29:09.297965 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:09.297619 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:09.312584 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:09.312521 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" podStartSLOduration=1.3125017159999999 podStartE2EDuration="1.312501716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 21:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:29:09.312070843 +0000 UTC m=+3084.660380184" watchObservedRunningTime="2026-07-10 21:29:09.312501716 +0000 UTC m=+3084.660811046" Jul 10 21:29:10.402956 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:10.402931 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-drs4v_37aefdb2-595e-4f19-adc4-da96832e6150/dns/0.log" Jul 10 21:29:10.420910 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:10.420884 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-drs4v_37aefdb2-595e-4f19-adc4-da96832e6150/kube-rbac-proxy/0.log" Jul 10 21:29:10.575879 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:10.575831 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mn55h_92929035-69df-4c3f-8e04-9d272ae67909/dns-node-resolver/0.log" Jul 10 21:29:11.118192 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:11.118162 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p9p6r_71a681fe-80db-4415-acc7-c3c0417e4157/node-ca/0.log" Jul 10 21:29:12.060638 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:12.060604 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qn694_32575539-5873-49c4-8da6-9e1aba76c3a6/istio-proxy/0.log" Jul 10 21:29:12.275795 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:12.275755 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nl4qq_daca5d24-2543-4659-bd4f-571d9e41e545/discovery/0.log" Jul 10 21:29:12.345354 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:12.345225 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/0.log" Jul 10 21:29:12.405213 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:12.405181 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/1.log" Jul 10 21:29:12.503482 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:12.503440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69db686bdd-kc8rg_fbafa86c-d243-443d-bfcd-c78189d42283/router/0.log" Jul 10 21:29:13.210991 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:13.210948 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w566r_275d5012-3f9b-40e6-8517-d5be9077b5dd/serve-healthcheck-canary/0.log" Jul 10 21:29:13.784405 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:13.784375 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xjq5x_8201e185-c37c-4de1-a2d5-f7dd79598fa6/kube-rbac-proxy/0.log" Jul 10 21:29:13.802328 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:13.802301 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xjq5x_8201e185-c37c-4de1-a2d5-f7dd79598fa6/exporter/0.log" Jul 10 21:29:13.822397 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:13.822375 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xjq5x_8201e185-c37c-4de1-a2d5-f7dd79598fa6/extractor/0.log" Jul 10 21:29:15.313439 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:15.313416 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bhd5q/perf-node-gather-daemonset-ktlch" Jul 10 21:29:16.598130 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:16.598103 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-m6g28_3061a25c-4747-4ff0-98b4-1a8776efc82f/manager/0.log" Jul 10 21:29:16.624846 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:16.624821 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6558bffdbc-m6g28_3061a25c-4747-4ff0-98b4-1a8776efc82f/copy-manifests/0.log" Jul 10 21:29:18.121957 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:18.121924 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66b48cb7bf-pk6s7_6fe05a50-dcd5-446c-bfa8-4a7ecd6c9fc5/manager/0.log" Jul 10 21:29:18.455101 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:18.455065 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qn694_32575539-5873-49c4-8da6-9e1aba76c3a6/istio-proxy/0.log" Jul 10 21:29:18.654463 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:18.654432 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nl4qq_daca5d24-2543-4659-bd4f-571d9e41e545/discovery/0.log" Jul 10 21:29:18.696958 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:18.696930 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/0.log" Jul 10 21:29:18.759103 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:18.759076 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/1.log" Jul 10 21:29:18.831171 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:18.831149 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69db686bdd-kc8rg_fbafa86c-d243-443d-bfcd-c78189d42283/router/0.log" Jul 10 21:29:20.018431 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:20.018393 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qn694_32575539-5873-49c4-8da6-9e1aba76c3a6/istio-proxy/0.log" Jul 10 21:29:20.225740 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:20.225710 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nl4qq_daca5d24-2543-4659-bd4f-571d9e41e545/discovery/0.log" Jul 10 21:29:20.283510 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:20.283408 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/0.log" Jul 10 21:29:20.344680 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:20.344636 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6lprh_a1b1210f-ba9c-4541-9281-b9ae703ee904/istio-proxy/1.log" Jul 10 21:29:20.420842 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:20.420809 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69db686bdd-kc8rg_fbafa86c-d243-443d-bfcd-c78189d42283/router/0.log" Jul 10 21:29:23.369702 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:23.369671 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2ntld_20ab3bfc-e03c-4209-abb0-981521f1a488/kube-storage-version-migrator-operator/1.log" Jul 10 21:29:23.371384 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:23.371360 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2ntld_20ab3bfc-e03c-4209-abb0-981521f1a488/kube-storage-version-migrator-operator/0.log" Jul 10 21:29:24.681565 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.681536 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/kube-multus-additional-cni-plugins/0.log" Jul 10 21:29:24.699800 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.699777 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/egress-router-binary-copy/0.log" Jul 10 21:29:24.718436 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.718421 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/cni-plugins/0.log" Jul 10 21:29:24.735702 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.735646 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/bond-cni-plugin/0.log" Jul 10 21:29:24.753293 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.753272 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/routeoverride-cni/0.log" Jul 10 21:29:24.771888 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.771871 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/whereabouts-cni-bincopy/0.log" Jul 10 21:29:24.789236 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.789214 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhchj_53756faf-a958-4c95-a856-ad2593be1e1f/whereabouts-cni/0.log" Jul 10 21:29:24.824891 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.824871 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c5g85_dd11744a-1747-4d79-861b-4197fa608a0b/kube-multus/0.log" Jul 10 21:29:24.882344 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.882323 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fbz74_b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b/network-metrics-daemon/0.log" Jul 10 21:29:24.898863 ip-10-0-136-104 kubenswrapper[2577]: I0710 21:29:24.898846 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fbz74_b577ecb4-73e4-4cd5-b1a3-b97203f6fe0b/kube-rbac-proxy/0.log"